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A Factor-Analytic Study of Planning Abilities 


R. M. Bercer,' J. P. GuiLrorp, AND P. R. CHRISTENSEN! 
University of Southern California 


I. INTRODUCTION 


His sTupyY is the fifth in a series of in- 
designed to explore 
abilities considered to be important in 
the successful performance of high-level 
personnel.? For the first four studies in 


* Now at System Development Corporation (for- 
merly System Development Division of the RAND 
Corporation). 

* Under Contract N6onr-23810 with the Office 
of Naval Research, The ideas expressed in this 
paper are our own and do not necessarily rep- 
resent the opinions of the Office of Naval Re- 
search. Reproduction in whole or in part is per- 
mitted for any purpose of the United States Gov- 
ernment. Of the authors of this paper, J. P. Guil- 
ford is responsible investigator and director of 
the project, P. R. Christensen is assistant director, 
and R. M. Berger was in direct charge of this 
particular study. For a fuller discussion of this 
study see references no. 2, 4, and 5. 

P. R. Merrifield participated in the develop- 
ment of the ordering hypothesis and contributed 
greatly to the construction of items and the de- 
vising of scoring methods. 

We are indebted to the following persons in 
the University of Southern California depart- 
ments indicated for making available to us classes 
for preliminary experimental testing: Dr. Con- 
stance Lovell, Psychology; Dr. E. C. McDonagh, 
Sociology; Dr. D. W. Lefever and Mr. R. A. 
Jones, School of Education. In addition we grate- 
fully acknowledge permission for test use and/or 
adaptation by the following authors and publish- 
ers: R. L. Thorndike (test 5); R. B. Cattell and 
the Psychological Corporation (test 6); L. G. 
Humphreys (tests 9, 12, 33, 34, 35); D. C. Adkins 
and Press Features, Inc. (test 21); D. C. Adkins 
and the AGO (test 22); and Irving Lorge (test 
$2). Tests 37, 39, 49, 50, and 52 were parts of the 
Aircrew Classification Battery. 

We are very much indebted to the staff of the 
Personnel Research Laboratory, HRRC, Lack- 
land AFB, Texas, for making the testing possible 
and for the expert administration of a long and 
demanding test battery. In particular we wish to 


the series see (3, 6, 10, 13). The purpose 
of this study was to isolate and define the 
abilities involved in planning perform- 
ances. We were especially interested in 
planning as it enters into the activities 
of supervisory, military, and research per- 
sonnel. It is in the planning perform- 
ances of these groups that we have looked 
for clues as to hypotheses of ability re- 
quirements that are common in almost 
any kind of planning regardless of vo- 
cational context. 


There is little psychological literature bearing 
directly on planning; the most useful were the 
reports of previous factor analyses. A planning 
factor was found in the AAF Aviation Psy- 
chology Research Program but was not clear- 
cut as a very general or fundamental ability of 
planning or foresight (9g). Analyses were made 
of two special AAF batteries of foresight-and- 
planning tests, both of which included a small 
number of planning tests plus reference tests. 
The planning tests helped to define two new 
factors: a planning factor and an integration 
factor. The agreement between the two analyses 
was not very strong with respect to the new fac- 
tors. 

Two alternative hypotheses were advanced in 
Report No. 5 of the AAF Psychology Program 
Research Reports (g) to explain the nature of 
the planning factor. One hypothesis was that 
it might represent a type of ideational fluency, 
i.e., with the idea that the man who can think 
of more solutions per unit of time would have 
an advantage with some of these tests. This ex- 
planation does not hold very well, since the 
production of ideas is no more important for 
performance on the tests that were loaded on 
the factor than for other tests that did not have 
significant loadings. The second hypothesis is 


thank Dr. Lloyd G. Humphreys, Director of Re- 
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istrators and proctors. 


2 R. M. BERGER, J. P. GUILFORD, AND P. R. CHRISTENSEN 


that some form of visualization different from 
the manipulation type is required in tests of 
planning. Convincing evidence for this hypothe- 
sis is also lacking. 

In an analysis of the AAF Sheppard Field 
battery of tests, Guilford, Fruchter, and Zim- 
merman (7) found a planning factor, which they 
labeled “planning speed.” Most of the tests that 
appeared loaded on this factor had been analyzed 
in AAF studies discussed above. Due to the in- 
clusion of a large number of planning tests, the 
investigators felt the planning speed factor was 
probably more stable than the planning factor 
found in the previous AAF analyses. The new 
factor was termed planning speed, since the 
emphasis had shifted to tests in which speed 
plays a more important role. 

Lucas and French included several of the 
AAF planning and integration tests in a recent 
study (11). An integration factor corresponding 
to the AAF integration III was tentatively identi- 
fied, although the planning tests did not show 
up on the factor as they did in AAF analyses. 
The presence of two planning tests on another 
factor suggested it might be a planning factor. 
The evidence is not clear, however, since the 
loadings and communalities tend to be very low. 


II. 


Planning was conceived not as a uni- 
tary psychological function but as a term 
to cover certain classes of activities. On 
the basis of our considerations of plan- 
ning activities and previous studies, six 
major hypotheses were developed con- 
cerning the abilities involved. Four were 
formulated in terms of classes of activi- 
ties involved in the various stages of 
planning. These were: orientation, hy- 
pothesized as an ability to see an order or 
trend in a mass of information and to 
recognize the pertinent variables operat- 
ing in a situation; prediction, the ability 
to see beyond the material given; elabo- 
ration, the ability to produce ideas or 
representations of ideas that contribute 
to the development of plans; and order- 
ing, the ability to arrange bits of infor- 
mation into a meaningful series. Two 
additional hypotheses were constructed 
pertaining to the qualitative aspects of 
planning. They were: ingenuity, the 


ability to find new and different methods 
of accomplishing a task; and evaluation, 
the ability to detect limitations in plans 
and to determine relative importance of 
variables. 

Each major hypothesis indicates an ex- 
pected primary thinking ability. Each 
subhypothesis reflects a specific feature 
or variation or alternative conception of 
the primary-ability dimension. These hy- 
potheses and subhypotheses have served 
as the logical base for what was recog- 
nized as a continuing research program 
in which this study is an exploratory in- 
vestigation. From an operational point 
of view, the subhypotheses served as start- 
ing points for test ideas. 

The complete list of hypotheses, sub- 
hypotheses, and the tests (by number) de- 
signed to measure them follows: 


1. Orientation 

a. Sensitivity to order: seeing an order or 
trend in a relatively ambiguous situation 
(tests 1-3, see Section IIT). 

b. Recognition of variables: awareness of the 
pertinent variables operating in a situation 
(tests 4-5). 

. Prediction 

a. Extrapolation: seeing the effects of a spa- 
tial or temporal extension of a present 
trend (tests 6-8). 

b. Foresight: relating what is given to what 
lies ahead (tests g-12). 

3. Elaboration 

a. Specification: specifying the necessary de- 
tails (tests 13-15). 

b. Production of alternate methods: finding 
different arrangements of the same material 
that will fulfill the requirements of an 
adequate solution (tests 16-17). 

c. Symbolization: producing adequate repre- 
sentations of ideas (tests 18-19). 

. Ordering 

a. Temporal ordering: arranging steps in a 
time sequence (tests 20-22). 

b. Hierarchical ordering: arranging subject 
matter according to class and subclass (tests 
23-25). 

. Ingenuity 

a. New methods: finding new or uncommon 
procedures (tests 26-27). 

b. New applications: adapting methods to new 
situations (test 28). 


pee 
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6. Evaluation 
a. Importance of variables: determining rela- 
tive importance of variables (tests 29-30). 
b. Seeing deficiencies: detecting the shortcom- 
ings of a proposed procedure (test 31). 


Reference factors (previously known 
factors expected) are: planning (AAF 
analyses), integration III, verbal com- 
prehension, general reasoning, logical 
evaluation, originality, ideational flu- 
ency, associational fluency, adaptive flexi- 
bility, eduction of conceptual relations, 
judgment, visualization, symbol manip- 
ulation, and numerical facility. 

Included in the battery is a test that is 
necessarily complex in that it requires 
the examinees to devise a complete plan. 
The test is an adaptation of Lorge’s 
Planning Skills test. A detailed account is 
given of a military situation in which 
there are problems of morale and leader- 
ship; the examinee is required to write 
down the steps in a plan of his own mak- 
ing that will improve the situation. Fur- 


ther description of the test will be given 
in section III. It should be informative 
to note to what extent a score on a test 
of general planning will load on the 
various factors that emerge in the analy- 
sis. 


III. THe Test BATTERY 


Several considerations influenced the 
kinds of new tests constructed and kinds 
of scores used: 


1. Many of the tests require completion-type 
answers. It was felt that since some planning 
activities are necessarily productive, the measures 
of the abilities associated would best be made on 
the basis of productive efforts of the examinee. 

2. An attempt was made to develop tests 
which, although of the completion type, might 
be scored objectively. A minimum requirement 
for those to be scored subjectively was that it 
was possible to develop a reasonably reliable 
score. 

3. Separately timed parts were constructed 
where possible to permit the computation of 
meaningful alternate-forms reliability estimates. 

4. An effort was made to construct unique 


tests where possible, that is, tests of one common 
factor, except in a few cases where the task pre- 
sented is of necessity complex. 

5. Where possible, simple tasks and situations 
were used in order to minimize variances due 
to differences in personal experience. 

Most of the new or adapted tests that 
were to serve as experimental tests were 
pretested on groups of college students in 
order to determine adequate lengths, 
time limits, and instructions. Informa- 
tion derived from this pretesting was 
used in rejecting some of the tests and 
in revising and improving the 31 experi- 
mental tests to be included in the final 
battery along with 16 reference tests. In 
addition, several Air Force test scores 
from the classification battery were in- 
cluded in the analysis, also for reference. 

All tests included in the analysis are 
described in the next section. For con- 
venience, they are arranged in the order 
of the hypotheses rather than in the order 
of administration. For each test, the na- 
ture of the task is stated briefly, and 
wherever possible a sample item is given. 

It should be noted that the sample 
items are transparently easy, since they 
were used as part of the test instructions 
for orientation. With respect to difficulty, 
then, the sample items are not typical of 
the items in the test proper. 

The number of separately timed parts, 
the number of items per part, and the 
working time are indicated. Since the 
scoring procedure for many of the tests, 
especially those requiring subjective eval- 
uations of completion responses, may be 
of interest, a summary of the scoring pro- 
tocol is also given for most of such tests. 

The code numbers of the experimental 
tests and some reference tests developed 
at this laboratory indicate the following 
information: The first letter—P, R, C, or 
E—indicates planning, reasoning, creativ- 
ity, or evaluation, respectively. The sec- 
ond letter stands for a specific hypothe- 
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sis; in the case of planning, R means ori- 
entation, X means prediction, E means 
elaboration, O means ordering, I means 
ingenuity, and J] means evaluation. Next 
comes the number of the test under the 
hypothesis, usually in order of the sub- 
hypothesis listed: 01, og, etc. The last 
letter indicates the form of the test, A 
meaning the first form, B the second 
form, etc. Several experimental tests are 
uncoded; these are tests adopted or only 
slightly adapted from other sources. 


A. DESCRIPTION OF EXPERIMENTAL TESTS 


1. Orientation 
a. Sensitivity to Order 


Test 1. Matrix Order—PRO1A. Each 


A, mouse rat 

Answer: Animals become larger. 
B. century year decade 
Answer: Time unit becomes smaller. 


lion 


item is a g X 3 matrix of words. E (the 
examinee) is required to draw a straight 
line through the sequence of three words 
that are in the most meaningful order. 
The line may be drawn through any one 
of three columns, any one of three rows, 
or either of two diagonals. In addition, 
the direction of the order is indicated by 
arrowheads. An arrowhead at one end 
shows the temporal order of the sequence 
and arrowheads at both ends of the line 
signify a spatial or logical relationship 
that can go either way (see Fig. 1). 


Sample items: 


Fic. 1 


An order in time is found in sample A. 
Order in space is found in the diagonal 
of sample B. The double arrow in sample 
B indicates that the sequence “skin-shirt- 


coat” is as logical an order as “‘coat-shirt- 
skin.” 

Scoring: An item is counted correct 
only when both the correct line is drawn 
and the correct placement of the arrow- 
head is made. The total score is the num- 
ber of items correctly marked. 

Parts, 2; items per part, 15; working 
time, 18 minutes; maximum score (high- 
est possible score), go. 

Test 2. Seeing Trends—RIO6A. Each 
item consists of a group of words whose 
arrangement is not entirely uniform, E 
attempts to discover a trend from left to 
right and writes his description of the 
trend below the item. 


Sample items: 


cow elephant 


minute 


Note that in both items several re- 
versals of the trend occur between words 
taken as pairs and it is necessary to look 
beyond adjacent words for the predomi- 
nant trend. 

Scoring: Total score is the number of 
correctly marked items. Only one kind 
of response is keyed for each item. In 
some cases responses worded differently 
than the keyed ones are acceptable if the 
meanings are close or if the responses are 
negatively worded, 

Parts, 2; items per part, 10; working 
time, 10 minutes; maximum score, 20. 

Test 3. Sensitivity to Order—PRO2A. 
Each item consists of 5 words that are 
ordered in some way. Some of the items 
present the words in the right order, 
others have one word out of order. If the 
arrangement as given is correct, E marks 
a plus sign. If the words are not in their 
best order E is required to circle the 
word that is out of place and to put a 


pig 
day week second 
A. run want B. tank cloth 
read © sleep accident flare 
walk ish book mixmaster valley ¢ 
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check to show where that word belongs 
in the group (see Fig. 2). 
Sample items: 


A. key, 


B. accident, 


door, room, house + 


reporter,” newspaper, reader 


Fic. 2 


lock, 


In sample item A the objects are in 
good order. In sample item B the ar- 
rangement is not as good as it might be, 
since the presence of a reporter is usually 
necessary before the write-up of an acci- 
dent can be made. “Write-up” is circled 
and a check indicates that it belongs be- 
tween reporter and newspaper. 

Scoring: Only the items in which some 
change is to be indicated in the order 
are scored. The presence of correctly ar- 
ranged items that the examinee is re- 
quired to mark with a “plus” serves to 
keep the examinee from indicating 
changes in other items that also appear 
to be properly arranged to him. The 
items that are scored must have the ap- 
propriate word circled and the check 
correctly placed in order to receive credit. 
The total score is the number right. 

Parts, 2; items per part, 15; working 
time, 8 minutes; maximum score, 18. 


b. Recognition of Variables 


Test 4. Pertinent Questions—PRO3A. 
E is presented with a number of conflict 
situations in which additional informa- 
tion would be required for making a 
choice. E is asked to write four questions, 
the answers to which would serve as a 
basis for making a decision. 


Sample item: 


A student who has just graduated from college 
is offered positions in two different parts of the 
country. What four questions have to be con- 
sidered in making a choice? 

Possible answers: What would the work in- 
volve in the two jobs? What salaries would be 
paid? In what places are the jobs? What are the 
relative opportunities for the future? 


Other questions may be acceptable but 
they may not repeat previous questions 
and they must refer to different aspects 
of the problem. 

Scoring: One point is given for each 
response that refers to a separate aspect 
of the problem. The total score is the 
number of responses credited. 

Parts, 2; items per part, 4; working 
time, 12 Minutes; maximum score, 32. 

Test 5. Awareness of Variables—PRO- 
4A. This is a test developed by the 
Human Resources Research Laboratories 
at Bolling AFB. A problem is presented 
in which some decision is needed. Ac- 
companying the problem are a number 
of statements of facts that may be perti- 
nent to the problem. E’s task is to select 
a specified number of facts that seem 
most important in reaching a decision. 


Sample item: 


You are driving from Middleton to Warrens- 
burg. You have plenty of time for the trip. 
There are two possible routes: Route A and 
Route B. Which would be better to take? 

A. Route A is 105 miles long and Route B is 
115 miles long. 

B. Route A goes through 5 small cities on the 
way, while Route B goes through 3 some- 
what larger cities. 

C. Route A has 10 miles of road under construc- 
tion, with dirt surface and sections of one- 
way traffic. 

D. Route B is mostly winding, bumpy macadam, 
while Route A is mostly 4-strip concrete. 

E. Route A goes north of Lake Winepaw, while 
Route B goes south of the lake. 

Which two facts are the most important to 
consider in deciding which route to take? Circle 
the letters preceding the two statements se- 
lected. 


The best answers are generally agreed 
to be C and D. 

In the test itself, each problem has 15 
facts under it, and E is to select the five 
facts that are most important. 

Scoring: Expert opinion was used in 
determining the “most important” facts 
for each of the four problems; the facts 
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on which most experts agreed constituted 
the provisional key. Scores were obtained 
using this key and the results were sub- 
jected to item analysis. This led to cer- 
tain minor modifications resulting in the 
final key. The number of “important” 
facts in this final key varies from 4 to 5, 
depending on the empirical results. In 
scoring, a weight of 1 is given to each 
agreement with the key. The total score 
is the number of such agreements. 

Parts, 4; items per part, 1; elements 
per item, 15; working time, 12 minutes; 
maximum score, 18. 


2. Prediction 
a. Extrapolation 


Test 6. Series. This test is an adapta- 
tion from the items included in a part of 
R. B. Cattell’s A Culture-Free Test. Each 
item consists of a series of figures, which 
are varied in some systematic fashion. E 
has to select the figure that goes next in 
the series by choosing among six given 
alternatives (see Fig. 3). 


Fic. 3 


Scoring: Score is the number of items 
correctly marked. 

Parts, 2; items per part, Part 1-7, Part 
2-8; working time, 514 minutes; maxi- 
mum score, 15. 

Test 7. Effects—PXO1A. Given a state- 
ment of some present-day trend, E at- 
tempts to predict what the nature of the 
effects of this trend will be at a specified 
future time. Each item consists of the 
statement of a trend and provides room 
for writing down four possible effects. 
The trends given are realistic and cover 
many different areas in which there is 
common experience. 


Sample item: 


A. There have been more girls born in the 
last five years than boys. What effects will this 
have 20 years from now (in addition to the 
obvious effect that there will be more women 
than men)? 


Possible answers: More unmarried women than 
unmarried men. More rivalry among women for 
husbands. More women would take up career 
work. Agitation to allow polygamous marriage. 


Scoring: Four responses can be given 
to each item and each response is 
weighted either o, 1, or 2. If the response 
is a duplication of a previously given one 
or is not a possible effect of the trend 
given, it is weighted zero. Effects that are 
possible are weighted 1; an additional 
weight of 1, or a total response weight of 
2, is allowed if the effect is less obvious 
or is somewhat indirect. The total score 
is the sum of the weights. 

Parts, 2; items per part, 5; working 
time, 16 minutes; maximum score, 8o. 

Test 8. Consequences (Remoteness)— 
CFO6A. For each item in this test a 
change of an unusual nature is imposed 
which would disrupt the normal state of 
affairs. E is required to list as many dif- 
ferent consequences or results of these 
changes as he can. 


Sample item: 
What would happen if all national and local 
laws were suddenly abolished? 


Possible answers: People would no longer get 


parking tickets. There would be more killing. 
No more taxes. 


Scoring: There are two scores derived 
from this test: a low-quality score and a 
remoteness score.:For the low-quality 
score the number of direct responses is 
determined. Direct responses are those 
that describe direct or immediate conse- 
quences of the given changes. The low- 
quality score is used as a measure of ide- 
ational fluency (see test 44). The remote- 
ness score is the one used here as a meas- 
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ure of prediction. The score is based on 
the number of indirect or remote conse- 
quences listed. The individual responses 
are rated as direct or remote by compar- 
ing them with a comprehensive list of 
model consequences that had been shown 
to be reliably weighted in a previous 
scoring for the creativity analysis. Every 
response is thus scored either 1 or o for 
remoteness. The total score is the sum 
of the weights. 

Parts, 4; items per part, 1; working 
time, 8 minutes; maximum score, unre- 
stricted (U). 


b. Foresight 


Test 9. Competitive Planning. This is 
an Air Force test modified slightly for 
purposes of this study. The test is based 
on the familiar “completion-of-squares” 
game. Each of two hypothetical oppo- 
nents (Black and White) tries to complete 
as many squares as possible in figures that 
are incomplete. E has to plan the moves 
for both opponents so that each achieves 
the maximum number of squares pos- 
sible. Black always makes the first move, 
filling in one side of an incomplete 
square. Each time Black or White com- 
pletes a square, he has to make one ad- 
ditional move. E must look ahead in 
many cases in order to see that a move 
giving an immediate gain should be 
passed over in favor of one making pos- 
sible greater gains for future moves. 

This is an answer-sheet test and an 
answer legend provides five alternative 
outcomes for the number of squares com- 
pleted by the opponents, Black and 
White (see Fig. 4). 


Sample item: 


Answer Legend 


Black 0 White 4 
Black 1 White 3 
White 2 
White 1 
White 0 


Black 2 
Black 3 
Black 4 


Fic. 4 


Making the best moves possible for 
their individual interests, Black and 
White complete this sample problem in 
the following manner: as always, the first 
move is made by Black. No matter which 
side Black fills in, White is able to com- 
plete two squares immediately. After 
completing the second square, White fills 
in a side of one of the remaining squares. 
This enables Black to complete the re- 
maining squares so that the final result 
becomes two squares for Black and two 
squares for White. The answer marked, 
therefore, should be C. 

Scoring: The score is the number of 
items right minus the number wrong di- 


WwW 
vided by four (R — —). 
4 


Parts, 2; items per part, 20; working 
time, 20 minutes; maximum score, 40. 

Test 10. Symbol Grouping—PXO24. 
Each item consists of a line of X’s, —’s, 
and O’s presented in a scrambled order. 
The task is to make successive moves of 
groups of one or more adjacent symbols 
to achieve the correct order. The desired 
order for all items is first all the X’s, next 
all the ~’s, and last all the O’s. The score 
depends not only on finding a solution 
but also on making as few moves as pos- 
sible. Since there are a variety of ways of 
getting to the correct order, foresight is 
hypothesized to be demonstrated when 
the most economical procedure is 
adopted. E indicates each move by cir- 
cling the symbol or symbols to be moved 
and placing a checkmark at the place 
they are to go. This is repeated for each 
new arrangement resulting until the so- 
lution is reached (see Fig. 5). 

Scoring: A weight of 2 is given when 
the solution is reached in the minimum 
number of steps, and a weight of 1 is 
given for the minimum number of steps 
plus one more step. Two steps or more 
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Sample item: 


Given: X - O X X - O X -, the following moves lead to the 
correct order. 


xox x - 
x x¥ o& xo 
xxxX---00 


Fic. 5 


beyond the minimum is felt to indicate 
little or no foresight and is therefore 
given no credit. Unsolved and un- 
attempted problems are also not credited. 
The total score is the sum of the weighted 
item scores. 

Parts, 2; items per part, 8; working 
time, 10 minutes; maximum score, 32. 

Test 11. Contingencies—PXO3A. In 
each item an object is named that is to 
be considered in connection with a given 
situation. E’s task is to state a possible 
condition that might require the use of a 
given object in that situation. Four ob- 
jects are presented with each item in the 
test. 


Sample item: 


Sally and Jane are going out to a farm to pick 
berries for the day. Their pay will be half the 
berries they pick. Included in the things they 
will take are the objects listed below. For each 
object, state a condition or circumstance that 
may arise requiring the use of that object. (Do 
not merely give the use of the object, but write 
why it might come into use.) 


Ointment If bitten by insects or scratched. 

Pins For possible rips in the clothes. 

Scoring: Each written response is 
scored as either acceptable or unaccept- 
able. A response is acceptable if a plaus- 
ible condition is given that might arise 
and require the use of the object. The 
total score is the number of acceptable 
responses, 

Parts, 2; items per part, 20; working 
time, 12 minutes; maximum score, 40. 

Test 12. Route Planning—Cl411AX. 
This is an adaptation of an Air Force 
test. For each item E must trace a path 
from the corner of a printed maze to a 


goal box in its center. There are four 
item numbers, one at each corr - of the 
maze, that serve as starting points. The 
task is to locate the one point through 
which one must pass in going to the goal. 
Letters mark the points on the various 
routes between the starting point and 
the goal. When E finds the one letter 
which he must pass through in order to 
reach the goal he marks the letter on the 
special answer sheet provided (see Fig. 6). 


Sample item: 


In going from gi: to the goal, you may 
pass through either A or C; however, you 
must pass through D; therefore, D is the 
right answer. Answers for other items 
are: 92--B, 93—G, and 94—-D. 

Scoring: R — W/2. 

Parts, 5; items per part, 8; working 
time, 17 minutes; maximum scores, 40. 


3. Elaboration 
a. Specification 


Test 13. Planning Skills II—PEO2A. 
This test is an adaptation of Irving 
Lorge’s Planning Skills Test (the latter is 
employed as a test of general planning 
and described as test 32). E is allowed 
five minutes to read a description of the 
conditions prevailing at some hypotheti- 
cal military, weather-station outpost in 
the Arctic. The conditions for the most 
part entail the effect of the rigors of Arc- 
tic temperature and isolation on person- 
nel assigned to the stations. After reading 
the several paragraphs involved, E is 
given the signal to turn to the first part 
of the test, which requires that he write 
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as detailed a plan as possible on some 
given aspect of the problem situation. 
For the second half of the test another 
particular aspect of the problem situation 
is given and E is to write down all the 
details necessary for coping with that 
part of the problem. 

Scoring: Each detail listed, if specific 
enough, is given credit. The total score 
is the number of acceptable details. 

Parts, 2; items per part, 1; working 
time, 16 minutes; maximum score, U. 

Test 14. Planning Elaboration—PEO- 
1A, Each part consists of a statement in 
which some plan of activity is briefly out- 
lined. E’s task is to fill in as many of the 
details as he feels necessary to make the 
plan work. Each point or detail specified 
is written on a separate line. 

Scoring: The total score is the number 
of details specified. 

Parts, 2; items per part, 1; working 
time, 16 minutes; maximum score, U. 

Test 15. Figure Production—PEO 3A. 
Each item in this test consists of markings 
of one type or another. The markings as 
given have no particular meaning. E’s 
task is to add to what is given as much as 
he feels necessary to produce a complete 
and meaningful figure. E can draw any 
figure he wishes as long as the given 
markings are definitely incorporated in 
what he makes (see Fig. 7). 


S 


this ++» oF this 
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Sample item: 
Given: oO 


Possible answers: 


Scoring: A total of 140 sample re- 
sponses to the stimuli were categorized 
by three raters into a 5-point equal- 
interval scale of “elaborateness.” The re- 
sponses on which all raters agreed were 
used as models for their respective cate- 
gories, Categories were assigned weights 
from o (least elaboration) to 4 (greatest 


elaboration). The remaining responses 
were weighted according to the category 
in which they were placed by a qualified 
rater. Score is the sum of the weights over 
all responses. 

Parts, 3; items per part, 4; working 
time, 9 minutes; maximum score, 48. 


b. Production of Alternate Methods 


Test 16. Alternate Methods—PEO4A. 
In each part a task is described that may 
be carried out in a number of nearly 
equally effective ways. E attempts to write 
down as many as six different ways of dis- 
tributing the work and assigning person- 
nel in order to accomplish the task. 


Sample item: 


A house located near a stream is on fire. 
Twenty men, each carrying a bucket, arrive to 
help put out the fire. The house is about 20 
yards from the stream. In how many ways could 


you organize this bucket brigade to deal with 
the fire? 


Possible answers: a. Form a line of 20 men; 
the first man fills a bucket and it is passed from 
man to man; the last man throws the water on 
the fire and runs back with the empty bucket. 

b. Form two lines of 10 men. The filled 
buckets are passed up one line and the empty 
ones back down the other. 


c. Assign 5 men to fill the buckets, 10 men to 
run with the buckets, and 5 men to throw the 
water where it is needed. 

Scoring: In order to be counted as an 
acceptable response, each method listed 
must be practicable, employ all or most 
of the given material, and be in some way 
different from the other methods pro- 
duced. The total score for the four items 
is the number of acceptable responses. 

Parts, 4; items per part, 1; working 
time, 14 minutes; maximum score, 24. 

Test 17. Match Problems II—~PEOS5A. 
This test is a varied form of the Match 
Problems test to be described in a later 
section (test 46). These tests utilize the 
match-stick problems in which groups of 
matches are laid out to form patterns of 
squares or triangles. E must find a way 


10 R. 


of taking away a specified number of 
matches to leave a certain number of 
squares or triangles remaining. In test 17 
E has to find several different solutions 
for each problem. That is, different pat- 
terns of matches must be removed to 
leave the specified number of squares or 
triangles. Three or four different ways of 
solving each problem are possible (see 
Fig. 8). 


Sample item: 


TAKE AWAY 5 
MATCHES AND 
LEAVE 6 
TRIANGLES 


by, Bs 
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The matches to be taken away are in- 
dicated by crossing them out with short 
heavy marks. In the sample item above 
the solution is reached by removing 5 
matches in the four different patterns in- 
dicated. 

Scoring: Credit is given for each cor- 
rect, different pattern. The total score is 
the number of acceptable solutions. 

Parts, 2; items per part, 5; working 
time, 20 minutes; maximum score, 38. 


c. Symbolization 


Test 18. Symbol Production—PEO6A. 
In this test E must produce symbols of 
his own invention to represent given ac- 
tivities and objects. In order to be accept- 
able the symbol produced must be a 
meaningful one for the action and the 
thing presented (see Fig. 9). 

E is instructed not to draw stick fig- 
ures or cartoon figures performing the 


Sample item: 

Symbolize the underlined words: 
Ring the bell 1) (2) 
7 $8 
Open the door ® @) 

3) 4) ant 
Look into the room (s) (6) 
m 
Close the window 

( 

Fic. 9 
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acts but to try to symbolize the actions 
themselves. 

Scoring: Obviously a wide variety of 
symbols may be produced in this test. A 
response is judged as an acceptable sym- 
bol if some relation, however remote, 
can be seen between it and the object or 
action to be symbolized. Cartoon figures 
are not acceptable unless there is addi- 
tional material that can stand alone as a 
proper symbol. The same symbol used 
for different items is scored only on its 
first occurrence. The total score is the 
number of acceptable symbols. 

Parts, 2; items per part, Part 1-31, Part 
2-30; working time, 10 minutes; maxi- 
mum score, 61. 

Test 19. Line Drawing—PEO7A. This 
is a test developed by J. P. and Ruth B. 
Guilford as a prognostic test of creativity 
in design (7). E is given a list of adjec- 
tives for each of which he is to draw a 
simple line that expresses the meaning of 
the adjective. Any kind of line may be 
drawn, such as straight, curved, in the 
form of a wave, or an angular pattern. 
The test is not a speeded one; everyone 
is allowed to finish. 

Scoring: Each line is scored for form, 
direction, and type. Lines are described 
as to form in four ways: wave, curve, 
angle, and straight line. The direction is 
scored in three ways: horizontal, up (up- 
ward slant), and down (downward slant). 
The type may be heavy, medium, or 
light. This classification gives ten cate- 
gories within which to classify each line 
in the test, i.e., four for form, three for 
direction, and three for type. Using the 
responses from half of the examinee 
group, the frequency with which each 
line fell into each of the 10 categories was 
tabulated. The frequencies were trans- 
lated into sigma values above and below 
the mean and these values, adjusted by a 
shift of zero point to make them all posi- 
tive, were used as the weights. The score 
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for each line is the sum of the weights 
for form, direction, and type. The total 
test score is the sum of the line scores. 
Parts, 1; items per part, 24; working 
time, 10 minutes; maximum score, 242. 


4. Ordering 
a. Temporal Ordering 


Test 20. Temporal Ordering—POO1rA. 
In this test E is given a short description 
of a project and a list of some of the steps 
involved in the project. The steps are not 
in the best time order as listed. Following 
the list are questions that concern the 
proper order of the steps in different 
parts of the list. E indicaics the answers 
by filling in the letters corresponding to 
the appropriate steps. 


Sample item: 


Fixing a Flat Tire: 

While you are driving along a busy highway 
late one dark night, your right rear tire goes flat. 
The steps described below are some of those to 
be performed. 

a. Set out flares that had been stored in glove 
compartment. 

. Tighten nuts on wheel. 

Block wheels so car won't roll. 

Raise car with bumper-jack. 

- Replace flat with spare. 

Unlock the trunk to get wheel blocks. 

. Take bumper-jack out of trunk. 

In the blanks left in the following statements, 
enter the letters indicating the appropriate step 
or steps. 

1. The steps that should precede step d include: 

a, ¢, f, g- 

2. The first two steps of this plan, in order, are 

a, f. 

3- The last step of this plan should be step b. 


Scoring: The examinee had to decide, 
in general, which elements preceded and 
which followed the reference elements 
for a particular question. A weight of 
+1 is given each element properly 
placed. To correct for “padding,” against 
which the examinees were forewarned, a 
weight of —1 is given each element in- 
correctly placed. Where the proper order 
was requested, bonus points are added 


for responses in which the elements 
within a response are correctly ordered; 
points are deducted if the examinee’s 
order is too far from the correct se- 
quence. The score is the sum of the 
weights over all elements of the re- 
sponses. 

Parts, 4; items per part, 8; working 
time, 16 minutes; maximum score 75. 

Test 21. Picture Arrangement. This is 
Dorothy C. Adkins’ test based on se- 
quences from the comic strip known as 
“Louie,” which has been adapted with 
the permission of Press Features, Inc. 
Each item consists of a separate sequence 
in which the pictures are disarranged. 
E is required to number the pictures 
to produce what he 9 the most 
sensible order (see Fig. 1 


item: 
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To show how the pictures should be 
arranged in order to make sense, a “1” 
has been written under the picture ha 
should come first, a “2” under the pic- 
ture that should come pllOreen and so on. 

Scoring: The total:score is the number 
of correctly ordered sequences, i.e., items. 

Parts, 2; items per part, I-8, II-12; 
working time, 7 minutes; maximum 
score, 20. 

Test 22. Sentence Order—RPO4B. This 
was adapted from a test in Dorothy C. 
Adkins’ North Carolina battery of rea- 
soning tests. Each item consists of three 
sentences, which are to be arranged in a 
sensible order. E indicates the order by 
numbering the sentences. 


Sample item: 
2 She bought some food at the market. 
3 She returned home and cooked some of the 
food she had bought. 
z She went to the market. 


* 
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Scoring: The total score is the number 
of items in which the sentences are cor- 
rectly ordered. 

Parts, 2; items per part, 10; working 
time, 6 minutes; maximum score, 20. 


b. Hierarchical Ordering 


Test 23. Outlining I—POO2A. In this 
test a series of statements of differing 
degrees of generality are given and E’s 
task is to arrange the given statements 
so that the more particular (less general) 
statements are grouped under the more 
general statements with which they have 
most in common. One statement, the 
most general of all, is to be designated 
as the title. 


Sample item: 


Cats are useful as mousers. 

Dogs are fine as guardians. 

Cows give milk. 

. Animals are useful in a variety of ways. 

. Some pets are useful as well as being enjoy- 
able. 

. Sheep furnish wool, 

G. Farm animals are essential to our way of life. 


Instructions: Put the letter for each of the 
statements above in one of the spaces below so 
as to make the best outline. Use all statements; 
use each statement only once. 


Title D 
IG 


Il. E 


=O 


In the test proper the context of the 
hierarchical ordering task involves the 
characteristics of eggs as a food. 

Scoring: Weights are assigned to each 
element on the basis of its being grouped 
with the other elements with which it 
properly belongs and on the basis of its 
being at the proper level of generality 
as intended by the constructors of the 
item. Score is the sum of weights over all 
elements of the response. 


Parts, 1; elements per part, 16; work- 
ing time, 5 minutes; maximum score, 
21. 

Test 24. Outlining II—POO2A. This 
test was administered as Part II of the 
Outlining Test, for which the description 
appears above (test 23, Outlining I). The 
context of the hierarchical ordering task 
is the planning of a club meeting. 

Scoring: See test 23. 

Parts, 1; elements per part, 16; work- 
ing time, 5 minutes; maximum score, 21. 

Test 25. Word Matrices—CYOS5A. In 
this test E is given several word squares, 
each of which is composed of three hori- 
zontal spaces and three vertical spaces. 
Two or three of the spaces have already 
been filled in with words which set the 
pattern for filling in the rest of the 
spaces. E’s task is to arrange the words 
given below each square in their proper 
places so that the words of each row of 
the square will show the same relation- 
ships. 


Sample item: 


minnow 


net 
rod 


lake, whale, ocean, bass, harpoon, pool. 
The proper arrangement is: 


minnow pool net 
bass lake rod 
whale ocean harpoon 


The pattern in each of the horizontal 
lines is: type of fish, place it is usually 
found, and instrument with which it is 
caught. Each of the vertical columns is 
arranged so that all the items in each 
column are of a similar nature. For ex- 
ample, all the items in the first column 
are names of fish. 

Scoring: The total score is the number 
of matrices in which all words are cor- 
rectly arranged. 

Parts, 1; items per part, 12; working 
time, 8 minutes; maximum score, 12. 
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5. Ingenuity 
a. New Methods 

Test 26. Unusual Methods—P103A. 
Each item involves a problem which is 
ordinarily handled in ways that are fa- 
miliar to everyone; E’s task is to find 
two different and unusual ways of dealing 
with the problem. One restriction is that 
the ideas must be usable ones, although 
they need not be the best or as efficient 
as the methods already in use. 

Sample item: 

Suggest two unusual methods for accomplish- 
ing the following: 

1. Relieve the boredom and fatigue of doing 
steady work in a business or industry. (The usual 
method is to have regular breaks during which 
employees relax or take refreshments). 

Possible answers: Have employees change their 
tasks every hour. 

Have four-hour shifts instead of eight-hour 
ones. 

Scoring: The total score consists of the 
number of acceptable respouses. In order 
for a method to be acceptable it must be 
reasonably practical and not now in use 
to any appreciable degree. 

Parts, 2; items per part, 5; working 
time, 12 minutes; maximum score, 20. 

Test 27. Verifications—PIO2A. Certain 
phenomena of a physical or a biological 
nature are presented as facts to be veri- 
fied. E’s task is to find ways of demon- 
strating these facts. The phenomena are 
stated in nontechnical terms and satis- 
factory answers may also be phrased in 
general terms. For each phenomenon 
given, E is required to write down two 
different procedures for demonstrating 
or investigating its validity. 


Sample item: 


Give two ways of demonstrating that the inside 
of a large gas flame is not as hot as the outside 
parts. 

Possible answer: Hold stick in flame—outside 
parts burn sooner. 

Put thermometer in center of flame and then 
in outside part—temperature rises faster in sec- 
ond part. 


Scoring: The responses are weighted 
1 if the general method suggested is an 
acceptable one. An additional weight of 
1 or a total weight of 2 is given to re- 
sponses that are complete and sound. 
The total score is the sum of the weights. 

Parts, 2; items per part, 5; working 
time, 20 minutes; maximum score, 40. 


b. New Applications 

Test 28. Procedure Applications— 
PIO1A. In each part E reads a descrip- 
tion of a procedure as it is used in ac 
complishing some particular task. A gen- 
eralized statement of the procedure then 
follows in order to make sure that all 
examinees know the procedural principle 
that is used. E must then find four other 
tasks or instances where the same pro- 
cedure would apply. 


Sample item: 

In chemistry, one of the methods for getting a 
pure sample of some substance is by successive 
crystallizations. The substance is first put into 
solution, then crystallized out. This removes 
some impurities. The process is repeated several 
times; each crystallization removes additional 
impurities until only the pure substance is left. 
If we think of this method in the general sense 
of refining or purifying by repeating a process, 
in what other instances might this method be 
used? 


Possible answers: Athletic contests in which 
successive races or matches are used for elimina- 
tion purposes. 

Spelling bee—people who misspell words drop 
out until only the best speller remains. 


Rinsing clothes—each rinse takes out more 
soap until none remains. 

Scoring: Each procedure has two ma- 
jor aspects. A weight of 2 is given for 
each response in which both aspects are 
applicable; a weight of 1 is given for 
each response in which one is applicable; 
and a weight of zero is given for each 
response in which neither aspect is ap- 
plicable. The total score is the sum of 
the weights. 

Parts, 5; items per part, 1; working 
time, 12.5 minutes; maximum score, 40. 
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6. Evaluation 
a. Importance of Variables 


Test 29. Essential Operations—RPO3B. 
E is provided with a problem and five 
items of information, one of which is 
irrelevant to the solution of the problem. 
E indicates which is the irrelevant state- 
ment, but does not work the problem. 
This is a revised form of the test used 
in previous reasoning studies (3, 6), in- 
corporating certain revisions and addi- 
tions in the items that were intended to 
reduce the general-reasoning variance 
and maximize the judgmental aspects 
of the test. 


Sample item: 
Question: How many miles apart will the two 
boats be when they run out of gas? 


A. Boats X and Y each have 5 gallons of fuel. 
B. They cruise together toward the same point. 
C. Boat X uses 1 gallon of gas going 3 miles. 
D. Boat Y uses 2 gallons of gas going 1 mile. 


E. Boat Y is twice as heavy as boat X. 

The correct answer is E, since all the 
other statements, A, B, C, and D, pro- 
vide necessary information for finding 
the answer to the question. The fact 
that boat Y is twice as heavy as boat X 
does not help to answer the question. 

Scoring: R — W/4. 

Parts, 2; items per part, 10; working 
time, 16 minutes; maximum score, 20. 

Test 30. Ranking of Variables—PJO2A. 
This is an adaptation of the Awareness 
of Variables test described previously as 
test 5. Each item consists of a problem 
in which some decision has to be made 
and a number of facts that would be of 
aid in making the decision. E’s task in 
this test is to rank all the facts in order 
of their relative importance in reaching 
a decision. 

Sample item: The sample item given 
for test 5 is also used here. This time 
the statements are ranked in order of 
importance with 1 indicating the most 
important and 5 marking the least im- 


portant fact to consider. The ranks as- 
signed to the statements in the sample 
item of test 5 are: A-4, B-3, C-2, D-1, 
E-5. 

Scoring: The score for each item is 
determined from the sum of the differ- 
ences obtained between the keyed ranks 
and the ranks assigned by the examinee. 
By this system, the greatest possible error 
for each item composed of seven ele- 
ments is 24 points. (The maximum error 
would occur, for example, if the assigned 
rank order is the complete reverse of the 
keyed order). In order to make the higher 
scores correspond to the better responses, 
the sum of the obtained differences for 
each item is subtracted from 24. 

Parts, 2; items per part, 2; working 
time, 10 minutes; maximum score, 96. 


b. Seeing Deficiencies 


Test 31. Seeing Deficiencies—PJO1A. 
In this test E is presented with short 
descriptions of plans or activities that 
for some reason will not lead to the de- 
sired result. E’s task is to point out in 
what way the plan is faulty. 


Sample item: 


A growing city discovers pressing needs to im- 
prove both its streets and its sewer system. After 
due consideration, the council decides to work on 
the street improvement program first. What is 
wrong with this plan? 


Possible answer: Improving streets before sewer 
system would require the street improvement job 
be done twice. 

Scoring: One written response is called 
for in each item and this must corre- 
spond in meaning to the keyed answer. 
No penalty is made for difference in ex- 
pression of the answer. The total score 
is the number of items correctly an- 
swered. 

Parts, 2; items per par:, 10; working 
time, 20 minutes; maximum score, 20. 


7. General Planning 
Test 32. Planning Skills—PGO1A. This 
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is a test by Irving Lorge used in studying 
decision-making. The test requires E to 
devise a complete plan for meeting a 
practical problem. The problem is simi- 
lar to some of those encountered by the 
military in connection with human re- 
lations and morale. It involves the effect 
of the conditions at some hypothetical 
Air Force base in the Southwest upon 
the personnel assigned to the base. These 
conditions include large work load, lack 
of adequate working facilities, lack of 
recreational facilities, lack of facilities 
for dependents and relative isolation of 
the base. E’s task is to recommend a 
plan of action to solve the problems of 
low morale and low operating efficiency 
of the base. 

Scoring: The procedure used by Lorge 
in scoring the test is employed here. 
Lorge had defined, by means of a con- 
tent analysis, eight areas in which the 
responses could be grouped. Within each 
area there are general statements listed 
that serve to classify the responses as to 
quality and determine the weights that 


Sample item: 


should be attached to them. These 
weights were originally obtained by using 
a large sample of responses and demon- 
strating a high agreement among raters 
in setting the weights. The total score is 
the sum of the weights. 

Parts, 1; items per part, 1; working 
time, 15 minutes; maximum score, U. 


B. DESCRIPTION OF REFERENCE TESTS 
1. Planning (AAF Analyses) 


Test 33. Planning Air Maneuvers— 
CI4O8AX}3. This test assumes that E is 
a sky-writing pilot who must plan how 
to write two adjacent letters by flying 
the shortest possible path. The starting 
and finishing positions of the plane are 
shown, and the sharpest turn that the 
plane can make is indicated. With this 
information, and the large letters to 
be written presented in the test, E must 
select the most direct path in terms of 
the instructions and indicate the direc- 
tion he is traveling at each of the num- 
bered points (see Fig. 11). 


/ 
4 Sharpest Turn 


Start 


Start Finish ok 
- + 
/ 
! / 
\ 


Fig. 11 


(3) LA 
| 
Al {5 
A (2) 
B B (5) 
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By referring to the right-hand half 
of Fig. 11, it can be seen that the correct 
direction for 1 is A, for 2 is B, g is A, 
4 is B, and 5 is B. 

Scoring: Since the choice of a direc- 
tion at one of the numbered points 
affects the direction choice at subsequent 
decision points in the same pattern, it 
was decided to score the test on the basis 
of total patterns. Within each pattern 
of letters, all choices have to be correct 
before credit is given. The score is the 
number of correct patterns. 

Parts, 2; items per part, 10; working 
time, 16 minutes; maximum score, 20. 

Test 34. Planning a Cireuit—Cl4Or. 
This is another Air Force test of plan- 
ning. Each item consists of an electrical- 
circuit diagram with many intersecting 
wires and with several sets of terminals. 
E’s task is to trace the circuits and deter- 
mine at which pair of terminals a bat- 
tery should be placed in order to com- 
plete the circuit through a meter (see 
Fig. 12). 


Sample item: 
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Dots in the figure represent connections; that 
is, two wires are joined at that point. Where 
there is no such dot the insulated wires simply 
cross each other with no connection. Both poles 
of A are connected to the same point. This shorts 
out the battery, and will not make the meter 
work. B would also short out for the same reason. 
One pole of C is connected to one side of the 
meter and the other pole to the other side of 
the meter. C is therefore the correct answer. 


Scoring: R — W/4. 


Parts, 3; items per part, 12; working 
time, 12 minutes; maximum score, 36. 


2. Integration III 


_ Test 35. Code Analysis, Each item con- 
sists of a key row of five letters and 
numbers and below it five other similar 
rows which constitute the choices. E’s 
task is to select the choice row that con- 
tains a given number of symbols of the 
key row. The numbers 1 to 5, are inter- 
changeable with the letters A to E, re- 
spectively. In other words, A is the same 
as 1, B is the same as 2, etc. The first 
number, five or less (or its equivalent 
letter), in the key row indicates the num- 
ber of the symbols in the key row that 
the choice row should contain. 


Sample items: 


LFE53A Il. JGgA3B 
A.H 333A Ao 3 
B.335H3 B.J61C7 
c.HCiED cC.G7BD4 
DFiIiEES D.15 DCF 
E.133HD E27] 


Answers: I-D, 


Scoring: R — W/4. 
Parts, 2; items per part, 20; working 
time, 22 minutes; maximum scores, 40. 


3. Verbal Comprehension 


Test 36. G-Z Verbal Comprehension. 
This test is Part I (Short Form) of the 
Guilford-Zimmerman Aptitude Survey. 
It is a standard vocabulary test, present- 
ing in each item a given word and five 
alternative choices for the best synonym. 

Scoring: R — W/4. 

Parts, 1; items per part, 40; working 
time, 12 minutes; maximum score, 40. 

Test 37. Vocabulary Test—Cl604B. 
This is an Air Force test. It contains 
words of appropriate difficulty for men 
of approximately two years of college 
training. 

Scoring: R — W/4, converted to sta- 
nine value. 
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Parts, 1; items per part, 150; working 
time, 15 minutes. 


4. General Reasoning 

Test 38. Ship Destination—RGRO zB. 
Using a diagram and observing an in- 
creasing number of rules, E must deter- 
mine which of three given ports is near- 
est to a ship in a given position. In addi- 
tion to the rules of distance, the wind 
direction must be considered at the start 
of the test. Later the adequacy of the 
ports, the ocean current, and the initial 
heading of the ship must also be taken 
into account. All these conditions in- 
fluence distance according to given rules. 

Scoring: R — W/2. 

Parts, 1; items per part, 15; working 
time, 15 minutes; maximum score, 15. 

Test 39. Arithmetic Reasoning— 
CI206C. This is an Air Force test. The 
test consists of the setting up of an arith- 
metic solution to short problems. For 
example, if a plane uses a given amount 
of gasoline to fly a given distance, how 
much gasoline would be required to fly 
another given distance? In each item the 
correct response must be selected among 
five choices given. 

Scoring: R —W/4, converted to sta- 
nine value. 

Parts, 1; items per part, 30; working 
time, 35 minutes. 


5. Logical Evaluation 

Test 40. Logical Reasoning—RLROSA. 
This is a syllogism-type test. The items 
include categorical, hypothetical, and 
mixed syllogisms, not all of which are 
strictly formal. Two, three, or four state- 
ments are given in an item and the ex- 
aminee must decide whether the last one, 
or conclusion, follows from the preceding 
statements. 


Sample item: 
All bombers are heavier than air. All airplanes 


are heavier than air. Therefore, all bombers are 
airplanes. 


Answer: False. 


Scoring: R — W. 

Parts, 2; items per part, 15; working 
time, 10 minutes; maximum score, go. 

Test 41. Inference—RDO3B. Each 
item presents a statement of fact or 
opinion and five alternate conclusions. 
E indicates which of these can be drawn 
with most justification. The items are 
logically informal. 


Sample item: 
Most of the trees in the forest are green. 


A. There are no yellow trees in the forest. 
B. There are some yellow trees in the forest. 
C. Some of the trees in the forest are green. 
D. Green trees are the tallest in the forest. 
E. Pine trees are green. 


Answer: C. 


Scoring: R — W/4. 
Parts, 2; items per part, 8; working 
time, 10 minutes; maximum score, 16. 


6. Originality 


Test 42. Plot Titles (Cleverness)— 
CFO2B. Four brief story plots are pre- 
sented. E is instructed to write as many 
appropriate titles as he can for each plot. 

Scoring: Two scores are derived from 
this test; a low-quality score and a clever- 
ness score. The responses are rated clever 
or nonclever by comparing them with a 
comprehensive list prepared in connec- 
tion with previous studies. The clever- 
ness score is the total number of re- 
sponses judged clever. 

Parts, 4; items per part, 1; working 
time; 12 minutes; maximum score, U. 


7. Ideational Fluency 


Test 43. Plot Titles (Low Quality)— 
CFO2B. (See no. 42 above for description 
of test.) 

Scoring: Total number of low-quality 
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responses, i.e., responses not judged as 
clever. 

Test 44. Consequences (Low Quality )— 
CFO6A. (See no. 8 for description of 
test.) 

Scoring: Total number of low-quality 
responses. 


8. Associational Fluency 


Test 45. Controlled Associations II— 
CAFO2B. Each item consists of a com- 
mon word for which E is to list three 
synonyms. 


Sample item: 
ODD: unusual 
unique 
queer 
Scoring: Number of responses judged 
as sufficiently similar to the stimulus 
words. 
Parts, 2; items per part, 10; working 
time, 6 minutes; maximum score, 60. 


9. Adaptive Flexibility 
Test 46. Match Problems—CXO3B. In 
this test groups of matches are laid out 
to form patterns of squares and triangles. 
E’s task is to take away a certain number 
of matches and have a certain number 
of squares or triangles remaining. They 
must come out even with no extra 

matches left over (see Fig. 19). 


Sample item: 


TAKE AWAY 2 MATCHES 
AND LEAVE 2 SQUARES 


The correct matches to be taken away 
are indicated by crossing them out with 
short heavy marks as has been done in 
the figure above. Notice that if any other 
pair of matches is taken away an extra 
match is left over in addition to the 
two squares. 


Fic. 13 


Some of the test items are so con- 
structed that they are insolvable if the 
subject imposes unnecessary restrictions 
upon himself, e.g., that the squares or 
triangles be the same size. 

Scoring: Number of correct solutions. 

Parts, 2; items per part, 9; working 
time, 12 minutes; maximum score, 18. 

Test 47. Sign Changes—CXO4B. This 
test involves simple numerical operations 
in which E must substitute one arith- 
metic operation sign for another before 
performing the operation. 


Sample item: 


In the following problems, wherever you see: 
— replace it by x; + replace it by —. 


3—6=—18 
64+2=— 
4—3=— 


The answer for the second item should be 4 
and for the third item 72. 

Scoring: Number of correct responses. 

Parts, 3; items per part, 8, 16, 16; work- 
ing time, 2 minutes; maximum score, 32. 


ro. Eduction of Conceptual Relations 


Test 48. Verbal Analogies I-RCRO1C. 
This is a standard verbal-analogies form 
of test. The only new idea is an attempt 
to make the relationship between the first 
pair of words relatively difficult to grasp 
and the analogy relatively easy to fulfill. 


Sample item: 


1. electricity : energy : : green : ? 

A. color B. red C. grass D. trees E. foliage 
2. athlete : scholar : : hands : ? 

A. seclusion B. write C. study D. book E. mind 

Answers: 1-A, 2-E. 

Scoring: R W/4. 

Parts, 2; items per part, 15; working 
time, 12 minutes; maximum score, go. 


11. Judgment 


Test 49. Practical Judgment—ClI301C. 
This is an Air Force test. It requires the 
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selection of the best solution to a wide 
range of problem situations, mostly in a 
military setting; the items are of the 
type commonly called “work-planning” 
items. The answers are selected from 
groups of four or five alternatives. 

Scoring: 2R — W/3, converted to sta- 
nine value. 

Parts, 1; items per part, 30; working 
time, go minutes. 


12. Visualization 


Test 50. Mechanical Principles— 
C1I903B. This is an Air Force Classifica- 
tion test in which all items are presented 
pictorially. The task consists in the ap- 
plication of mechanical principles to the 
pictured problems. E has to select the 
correct response among three to five 
choices. For example, two ways of ex- 
tracting a nail from a board with a claw 
hammer are pictured, and the problem is 
to determine which of the two ways, if 
either, requires more force. 

Scoring: R — W/2 + 20, converted to 
stanine value. 

Parts, 1; items per part, 40; working 
time, 20 minutes. 


13. Symbol Manipulation 


Test 51. Symbol Manipulation. The 
items consist of a main statement com- 
posed of letters and symbols followed by 
several conclusions also stated in sym- 
bols. On the basis of the symbols given, 
E determines whether each of the con- 
clusions is true or false in relation to 
the main statement. 


Sample item: 


“L” means “larger “NL” means “not 


than” larger than” 
“§” means “smaller “NS” means “not 
than” smaller than” 


“E” means “equal to” “NE” means “not 


equal to” 


If x S y (x is smaller than y), then: 


1x Ey 

2.x NL y 

3x Ly 
Answers: 1. False 2. True 3. False 
Scoring: R — W. 


Parts, 1; items per part, 25; working 
time, 7 minutes; maximum score, 25. 


14. Numerical Facility 


Test 52. Numerical Operations— 
CI702B. This is an Air Force Classifica- 
tion test consisting of two parts: Part I 
—deciding whether answers to simple ad- 
dition and multiplication problems are 
right or wrong; Part II—finding the an- 
swer to simple subtraction and division 
problems. There are five choices. 

Scoring: R —3W, converted to sta- 
nine value. 

Parts, 2; items per part, I-97, II-78; 
working time, I—5, minutes, minutes. 


C. Tue Test ADMINISTRATION 


The planning battery was administered 
to entering aircrew trainees at Lackland 
Air Force Base, Texas, in January 1954. 
The sample number for which scores 
were obtained on all the tests was 364. 
The test administration was reported to 
have run smoothly and the examinees 
were assumed to have worked at the 
tests with fairly high motivation. The 
Aircrew Classification Battery had been 
administered just prior to the experi- 
mental battery. 


IV. TREATMENT OF THE DATA 


The classification tests were scored by 
the Air Force and scores converted to 
stanine values; the remaining tests were 
scored at this laboratory. 

The test reliabilities were computed 
on a subsample of 200 examinees. The 
values and types of the reliability esti- 
mates are given in Table 1 along with 
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TABLE 1 
Means, STANDARD DEVIATIONS, AND 
RELIABILITIES OF SCORES 


Test 


. Matrix Order 

Seeing Trends 

. Sensitivity to Order 
Pertinent Questions 
Awareness of Variables 


Hh 


Series 
Effects 


7 


Consequences ( t ) 
Competitive Planning 
Symbol Grouping 


PP Serre 


. Contingencies 

. Route Planning 

. Planning Skills I 

. Planning Elaboration 
. Figure Production 


. Alternate Methods 

. Match Problems II 
Symbol Production 

. Line Drawing 

. Temporal Ordering 


. Picture Arrangement 
. Sentence Order 

. Outlining (Part D 

. Outlining (Part ID 

. Word Matrices 


. Unusual Methods 

. Verifications 

. Procedure Applications 
. Essential Operations 

. Ranking of Variables 


31. Seeing Deficiencies 
. Planning Skills 
. Planning Air Maneuvers 
. Planning a Circuit 
. Code Analysis 


. G-Z Verbal Comprehension 
. Vocabula: 

. Ship Destination 

. Arithmetic Reasoning 

. Logical Reasoning 


. Inference 

. Plot Titles (cleverness) 

. Consequences (low t 

. Controlled ‘Associations Ii 


. Match Problems 

. Sign Changes 

. Verbal Analogies I 

. Practical Judgment 

. Mechanical Principles 


Manipulation 
umerical Operations 


* AF: alternate-form estimate; OE: odd-even estimate; Spear- 
man-Brown formula or Horst formula (where parts contained 
unequal numbers of items) was applied in AF and OE cases, 
except for tests 8, 42, 43, and 44; KR: Kuder-Richardson esti- 
mates; TR: Test-retest, A as a third letter refers to Air Force 
sources. The means and standard deviations of the Aircrew 
Classification tests were computed from stanine scores. The com- 
munality value is used as the lower-bound estimate of the reli- 
ability of test 32 because of the impossibility of making a valid 

test. 


estimate from what is essentially a single-item 2 


the means and standard deviations of 
the tests. 


Score distributions for most of the 
variables did not appear to be signifi- 


cantly deviant from normal and the raw 
scores were used in computing Pearson 
rs. The distributions of four of the vari- 
ables (tests 12, 21, 42, 43) were markedly 
skewed. For these variables normalized 
scores were used in computing Pearson 
rs. 

Two scores each were derived from 
the Plot Titles and Consequences tests, 
a low-quality and a cleverness score for 
the former, and a low-quality and a re- 
moteness score for the latter. In the 
correlation of one score of a test with 
the other score of the same test, the two 
cross correlations of separately timed 
part scores were computed and averaged 
in order to arrive at a correlation value 
relatively uncontaminated by any ex- 
perimental dependence of the two scores. 
For all other correlations in which these 
tests (Plot Titles and Consequences) were 
involved, the correlation coefficient used 
in the analysis was an average of those 
from two part scores. The correlation 
matrix is given in Table e. 

Seventeen centroid factors were ex- 
tracted.? The highest coefficient in each 
column of the correlation matrix was 
used as the estimate of the communality 
of the corresponding variable. After each 
extraction the highest residual value in 
each column was used as an estimate of 
the communality remaining. The load- 
ings of the last three centroid factors were 
all smaller than .20 but these factors were 
used in the later rotations in order to 
help clarify the other factors. 

The Zimmerman graphic, orthogonal 


* A copy of the centroid factor matrix has been 
deposited with the American Documentation In- 
stitute. Order Document No. 5269 from ADI 
Auxiliary Publications Project, Photoduplication 
Service, Library of Congress, Washington 25, 
D.C., remitting in advance $1.25 for 35 mm. 
microfilm or $1.25 for 6 x 8 in. photocopies. 
Make checks payable to Chief, Photoduplication 
Service, Library of Congress. 


20 
M Tu 
14 -70 
9 
13 
23 
9 
9 
| | 
9 
1 17 .65 
II 15 69 
12 31 +77 
13 Ir -73 
14 15 +57 
15 23 
16 Ir -59 
17 17 .65 
18 26 .86 
19 188 -63 
20) 37 .67 
at 15 -60 AF 
22 Ir -56 AF 
23 12 -57 KR 
24 Io] «50 KR 
25 4 -79 OE 
26 to +74 AF 
27 18 -72 AF 
28) 10 -63 AF 
29) -62 AF 
30 59 AF 
9 -58 AF 
13 -38 h? 
5 -48 AF 
20) -73 AF 
15 +71 AF 
36 20 .8r OE 
37 5 -83 TR-A 
38 6 -60 KR 
39 5 -79 OE-A 
40 10) -s8 AF 
41 6 -67 AF 
42 6 +43 AF 
43 12 +70 AF 
44) 13 AF 
45 a1 ~AF 
46 7 -56 AF 
47 a1 -69 AF 
48 12 +54 AF 
49 5 43 OE-A 
59 5 -82 OE-A 
st 15 -48 KR 
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method was used in rotating the axes 
(14). The criteria for rotations in the 
early stages were those of simple struc- 
ture and positive manifold. Where these 
criteria were in conflict, the rotation was 
guided by the criterion of simple struc- 
ture. After each axis had been rotated 
at least once, psychological meaningful- 
ness and agreement with previous well- 
known factor-analytic results became im- 
portant criteria. The final rotations were 
adjustive in nature in order to improve 
simple structure and positive manifold. 
The final rotated factor loadings are pre- 
sented in Table 3. 


V. INTERPRETATION OF THE FACTORS 


The factors are presented in the ap- 
proximate order of their familiarity and 
definiteness. The interpretations rest 
principally upon those tests with loadings 
of .go or greater, The test numbers, 
names, and factor loadings are listed 
preceding the discussion of each factor. 
In parentheses are loadings .30 or greater 
on other factors. Discussion of the refer- 
ence factors is omitted except where the 
factor is relatively less well-known. 


Factor A. Verbal Comprehension (V) 
37 Vocabulary -73 
36 G-Z Verbal 
Comprehension 71 
41 Inference Test 32 (.30 LE, .30 Or) 
45 Controlled Associations 
II 


Factor B. Numerical Facility (N) 
52 Numerical Operations .66 
47 Sign Changes 5l 
g9 Arithmetic Reasoning GR) 


Factor C. Visualization (Vz) 
50 Mechanical Principles .44 (.32 AX) 
12 Route Planning 32 (.38 PF, .34 AX) 


Factor D. General Reasoning (GR) 
38 Ship Destination 
35 Code Analysis 
39 Arithmetic Reasoning .g1 (.4 N) 
29 Essential Operations 31 (.g1 LE, .31 J) 
23 Outlining (Part I) 31 


44 
85 (33 PF) 


Factor E. Logical Evaluation (LE) 
40 Logical Reasoning 46 
29 Essential Operations 31 (.31 GR, .g1 J) 
41 Inference Test 30 (.32 V, .30 Or) 
Factor F. Ideational Fluency (IF) 
44 Consequences (Low 


Quality) 49 
43 Plot Titles 


(Low Quality) $7 


The two tests listed for factor F were 
included in the battery primarily as 
reference tests for ideational fluency. The 
factor was previously defined (13) as the 
speed of calling up ideas in a situation 
in which there is relatively little restric- 
tion, and quality does not matter. 


Factor G. Judgment (J) 
46 (.36 Or) 
43 (39 CF) 
38 


31 Seeing Deficiencies 
11 Contingencies 

27 Verifications 

29 Essential Operations 31 (.91 GR, .31 LE) 
49 Practical Judgment 30 

14 Planning Elaboration — 30 (.44 E) 


The judgment factor usually appears 
whenever “judgment” tests are included 
that are composed of items presenting a 
predicament and requiring the selection 
of the best solution. In the AAF analyses 
(g) the hypothesis offered for this factor 
was “the ability to weigh solutions and 
select the wisest and best one.” 

These results indicate that judgment 
is not bound to the multiple-choice test 
format. The individual himself can pro- 
duce solutions prior to weighing the 
merits of solutions. A fluent recall of 
experiences that are relevant to the prob- 
lem situation would provide the indi- 
vidual with a wide range of possible 
solutions from which to choose. In ex- 
amining a number of solutions to a practi- 
cal problem it is not unlikely that the 
best choice can be made when the ex- 
aminee considers the given solutions 
against the background of his experi- 
ence, placing them in a hierarchy of 
solutions that he produces. 
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TABLE 2 
Tue CorRELATION MATRIX® 


221 «147 
328 124 
369 «1 


070 


164 043 07 1 
249 +051 —029 —047 089 011 
072 058 114 003 —035 105-015 059 —004 177 099 008 


071 007 +075 #135 #196 188 123 152 150 094 


138 017 152 031 234 


068 —027 


124 —055 
142 070 


® Decimal points have been omitted, 


Factor H. Eduction of Conceptual 
Relations (CR) 


48 Verbal Analogies I 47 
3 Sensitivity to Order 46 


This factor emerged in the two reason- 
ing studies (3, 6). In both of those 
analyses, Verbal Analogies I was the 
leading test. The test was specifically 
designed to emphasize the discovery of 


meaningful relationships, and the inter- 
pretation of the factor was based upon 
this intent. 

The discovery feature is certainly im- 
portant in Sensitivity to Order also. In 
this test adjustments in the given orders 
of words are to be made when one of 
the words is seen to be out of order. 
Although the emergence of factor H in 


22 
253 180 121 166 191 117 161 175 O79 214 199 050 123 152 189 193 271 055 206 262 271 218 125 329 134 
253 137 232 138 264 273 219 171 065 266 132 121 226 144 223 173 209 168 251 157 251 169 216 248 223 
180 137 074 072 100 068 068 088 054 152 267 041 138 090 107 187 119 047 208 156 233 164 085 336 053 
121 232 074 170 132 626 340 062 036 482 163 467 465 257 516 136 316 023 214 160 219 065 223 141 409 
166 138 072 170 105 123 108 043 O18 132 048 136 104 064 085 055 113 113 144 181 102 117 085 130 O14 
191 264 100 132 105 162 167 167 128 105 146 049 130 166 220 195 263 128 256 206 201 151 131 285 086 
117 273 068 626 123 162 354 003 031 422 O71 488 464 371 542 089 340 O01 199 133 202 094 281 194 459 
161 219 068 340 108 167 354 121 050 333 066 292 345 214 342 038 248 O82 164 122 188 128 211 190 300 
175 171 088 062 043 167 093 121 153 092 232 039 101 052 136 303 116 062 156 152 093 113 068 106 061 
079 065 054 036 O18 128 031 050 153 088 160 062 037 024 107 170 024—004 048 096 087 059 055 078 134 
11. | 214 266 152 482 132 105 422 333 092 088 055 354 372 230 398 138 279 068 245 092 292 140 285 252 303 
12. | 199 132 267 163 048 146 O71 066 232 160 055 —065 039 016 121 372 134 128 248 266 110 094 040 202 071 
13. | 050 121 041 467 136 049 488 292 039 062 354 —065 479 287 336-032 117-014 093 023 168 095 200 164 344 
14. | 123 226 138 465 104 130 464 345 101 037 372 039 479 296 463 086 231 048 177 130 171 104 274 174 331 
15. | 152 144 090 257 064 166 371 214 052 024 230 016 287 296 290 056 280 024 156 204 163 056 172 106 304 
: 16, | 189 223 107 516 085 220 542 342 136 107 398 121 336 463 290 204 314 035 175 137 169 114 189 187 356 
17. | 193 173 187 136 055 195 089 038 303 170 138 372-032 086 056 204 100 —012 182 231 119 092 022 185 —001 
18. | 271 299 119 316 113 263 340 248 116 024 279 134 117 231 280 314 100 132 310 151 201 166 177 196 292 
19. | 055 168 047 023 113 128 001 082 062—004 068 128—014 048 024 035-012 132 148 084 177-010 061 151 014 
20. | 296 251 208 214 144 256 199 164 156 048 245 248 093 177 156 175 182 310 148 261 323 154 186 305 129 
21, | 262 157 156 160 181 206 133 122 152 096 092 266 023 130 204 137 231 151 O84 261 259 149 148 304 099 
22. | 271 251 233 219 102 201 202 188 093 087 292 110 168 171 163 169 119 201 177 323 259 166 187 342 086 
; 23. | 218 169 164 065 117 151 094 128 113 059 140 094 095 104 056 114 092 166-010 154 149 166 200 146 147 
24. | 125 216 085 223 O85 131 281 211 068 055 285 040 200 274 172 189 022 177 O61 186 148 187 200 204 144 
25. | 329 248 336 141 130 285 194 190 106 078 252 202 164 174 106 187 185 196 151 305 304 342 146 204 047 
26. | 134 223 053 409 014 086 459 300 061 134 303 O71 344 331 304 356-001 292 014 129 099 086 147 144 047 
27. | 242 285 169 376 186 202 421 292 060-010 397 145 200 323 220 339 104 206 062 242 082 172 144 196 259 280 
28. | 234 266 171 338 194 198 437 282 034—067 325 050 266 363 238 346 067 248 115 228 208 334 143 196 286 232 
29. | 264 254 227 158 269 204 213 222 198 067 271 187 120 191 029 213 215 105 084 268 136 282 163 250 347 063 
30. | 088 117 134 051 145 102—006 063 035-005 074 049 030 029—079 045 088 050 089 093 123 058 087 088 197 —099 
: 31. | 358 287 178 375 230 185 296 240 103 076 350 168 202 282 181 333 170 212 032 281 255 364 226 266 302 164 
32. | 105 191 149 290 053 099 340 218—034 044 306—O08 302 347 267 262-032 266 023 149 082 257 170 133 117 344 
33. | 148 110 158 O18 151 144 032—016 189 057 016 240-034 016 016 098 284 114-005 226 080 155 090 090 166 —040 
34. | 120 120 152 153 037 206 151 104 194 118 094 317 037 102 084 142 256 096 002 086 224 109 020-009 152 149 
35. | 230 112 146 O18 110 192 058 110 204 178 044 215 035 108—021 152 303 080 057 171 248 132 261 041 
36. | 276 204 225 188 276 186 248 200 035-045 301 096 145 200 081 163 052 172 189 389 155 229 290 028 
37. | 286 336 218 209 219 137 243 216 049-050 404 056 170 198 151 142 010 235 211 416 135 220 349 055 
38. | 203 248 230 104 148 319 119 166 258 O72 151 252 105 123 045 046 181 O84 174 191 264 265 192 123 292 113 
39. | 235 278 273 076 200 128 143 154 171 002 170 260 060 107 038 175 251 124 020 255 129 201 266 200 290 —039 
40. | 210 283 230 162 238 203 177 160 120 O19 122 245 044 178 051 208 188 066 065 238 146 262 103 147 298 —002 
41. | 316 314 195 253 226 160 288 230 116 026 332 191 158 195 190 254 181 217 127 324 255 342 148 209 357 176 
42. | 100 232 115 253 134 127 288 201 080-084 244 034 174 245 241 274 019 276 088 184 092 160 126 180 160 186 
43. |-012 022-103 286-076 016 294 140 O81 114 158—026 270 284 178 204 015 098 016-014 013—034 034 100-087 252 
44. | 009 062 022 252 034 029 304 240-034 050 184-039 243 248 205 244 082 140 032 036 013 020 020 040 014 216 
45. | 196 261 218 308 075 172 424 232 088 049 321 060 234 202 198 319 114 208 106 218—004 234 113 186 206 188 
46. | 162 165 183 098 089 254 087 078 277 115 047 395-054 050 161 139 455 111-010 228 263 108 133 082 219 077 : 
47, | 115 213 075 085-048 148 106 133 123 130 180 148 012 073 047 120 068 192 101 160 O71 130 144 122 184 069 
48. 301 196 368 169 153 224 183 168 #122 064 205 4170 179 #+%186 #118 213 201 116 044 264 198 286 188 208 382 —005 
49. | 195 105 196 129 112 023 183 008 
50. | 127 127 249 070 132 136 TY 
51. | 160 047 139 040 048 115 
52. | 084 142 
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TABLE 2 (Continued) 


059 
118 070 —047 —004 


88 
068 —134 102 154 117 


132 004 048 
179 148 223 
054 —088 —068 
026 —008 —002 
070 038 —048 


144 —109 —042 


Besse 


S325 


114 —076 —034 


187 031 032 326 194 055 
058 088 099 «129 «014 —012 

058 —117 —060 —153 — 
088 292 052 —013 —048 — 
187 095 306 202 079 1 
109 —052 00 247 «(128 
099 ©6052 093 
129 —117 2 
014 —060 —013 185 

055 —012 —153 —048 079 176 260 


147 036 —079 —023 154 114 1 
164 099 113 O17 139 120 425 089 


this analysis lends confirmation to the 26 Unusual Methods “82 (34 E) 

existence of this ability, the definition '° Figure Production 30 (37 E) 

has not been clarified, due to the paucity This seems to be the originality factor 

of tests with loading on the factor. that emerged in the analysis of creative- 
Factor I. Originality (O) thinking tests. In that study, Plot Titles 


18 Symbol Production 42 was the leading test on originality, and 

42 Plot Titles (Cleverness) .36 Consequences also had a significant load- 

2 Seeing Trends 35 in: 

(Remoteness) $4 (30 CF) Symbol Production was designed as a 


23 
242 234 264 088 358 105 148 120 230 276 286 203 235 210 316 100-012 009 196 162 115 301 195 127 160 084) 1. 
285 266 254 117 287 191 110 120 112 204 336 248 278 283 314 232 022 062 261 165 213 196 105 127 047 142) 2. 
169 171 227 134 178 149 158 152 146 225 218 230 273 230 195 115-103 022 218 183 075 368 196 249 139 O71/ 3. 
376 ©6338 «6158 051 375 290 018 153 018 188 209 104 076 162 253 253 286 252 308 098 085 169 129 070 040 102 | 
186 194 269 145 230 053 151 037 110 276 219 148 200 238 226 134-076 034 075 089-048 153 112 132 048 007 
202 198 204 102 185 099 144 206 192 186 137 319 128 203 160 127 016 029 172 254 148 224 023 136 115 075 
421 437 213-006 296 340 032 151 058 248 243 119 143 177 288 288 294 304 424 O87 106 183 087 028 O72 135 
292 282 222 063 240 218-016 104 110 200 216 166 154 160 230 291 140 240 232 078 133 168 048-028 058 196 
060 034 198 035 103 —034 189 194 204 035 049 258 171 120 116 080 081-034 088 277 123 122-027 047 114 188 
—010 —067 067-005 076 044 057 118 178-045-050 072 002 019 026-084 114 050 019 115 130 064-017-016 003 123] 10. 
397 325 271 074 350 306 016 094 044 301 404 151 170 122 332 244 158 184 321 O47 180 205 106 008-035 152] 11. 
145 050 187 049 168-008 240 317 215 096 056 252 260 245 191 034 —026—039 060 395 148 170 164 249 105 150] 12. 
290 266 120 030 202 302-034 037 035 145 170 105 060 044 158 174 270 243 234-054 012 179 043 051-015 0094] 13. 
323 363 191 029 282 347 016 102 108 200 198 123 107 178 195 245 284 248 292 050 073 Po 138} 14, 
220 238 029-079 181 267 016 084—021 O81 151 045 038 051 190 241 178 205 198 161 047 017) 15. 
339 346 «213 045 333 262 098 142 152 163 142 046 175 208 254 274 204 244 319 130 120 213 045 040 086 132] 16. : 
104 067 215 088 170-032 284 256 303 052 010 181 251 188 181 019 O15 O82 114 455 068 201 154 152 115 184) 17. 
206 «248 105 050 212 266 114 096 080 172 235 084 124 066 217 276 098 140 208 i111 192 116 110 100 035 068) 18. 
062 115 084 089 032 023-005 002 057 189 211 174 020 065 127 088 016 032 106-010 101 044 093 —026—021 042) 19. 
242 228 #268 093 281 149 226 086 221 328 369 191 255 238 324 184-014 036 218 228 160 264 175 090 177 152) 20. 
082 208 136 123 255 082 080 224 147 124 126 264 129 146 255 092 013 013 —004 263 O71 198 129-089 099 031] 21. 
172 334 282 058 364 257 155 109 171 389 416 265 201 262 342 160-034 020 234 108 130 286 216 039 168 118] 22. 
144 143 163 087 226 170 090 020 248 155 135 192 266 103 148 126 034 020 113 133 144 188-006 028 128 081] 23. 
196 196 250 088 266 133 090—009 132 229 220 123 200 147 209 180 100 040 186 082 122 208 056 O11 008 234) 24. 
259 286 347 197 302 117 166 152 261 200 349 202 290 298 357 160-087 014 206 219 184 382 183 068 124 142) 25. 
280 232 063 —099 164 344-040 149 041 028 055 113 —039-—002 176 186 252 216 188 077 069-005 008 —027—055 070 
365 286 140 302 223 096 148 065 330 310 156 237 289 284 204 138 086 318 118 064 260 147 305 031 —016 
365 229 «4098 380 282 065 122 O41 324 356 167 234 280 309 250 169 150 378 086 096 186 225 032 132 122 
140 098 088 089 4039 #059 O08 142 178 130 193 144 169 126 084 049 
302 380 376 089 257 188 232 155 399 381 285 281 284 374 304 156 063 334 299 106 122 086 
223 282 «4119 039 257 —016 062-001 217 249 133 047 059 171 259 4007 #093 «#128 105-103 073 021 
096 4065 #179 059 188 —016 215 240 097 #109 145 257 250 156 060 378 122 181 093 240 109 060 
148 122 097 008 232 062 215 110 078 074 172 060-005 130 054 258 O78 O84 117 182 093 —007 
065 O41 286 142 155-001 240 110 090 050 302 325 238 152 052 233 220 282-013 008 122 310 
330 324 358 178 399 217 097 078 090 782 243 327 #325 447 EE 398 119 089 473 300 166 297 088 
310 356 343 130 381 249 109 074 050 782 240 295 245 456 162 —074-—023 414 102 130 357 273 069 212 162 
156 167 325 193 285 133 145 172 302 243 240 246 4323 4205 124-074-087 174 200 142 299 124 163 170 169 
237 «234 «371 «4144 281 047 257 060 325 327 295 246 360 302 115 —048 —003 263 262 259 372 135 122 209 399 
289 280 402 169 284 059 250-005 238 325 245 323 360 370 «106 «178 «6192 129 
284 309 288 126 374 171 156 130 152 447 456 205 302 370 47 164) 41. 
204 250 102 068 132 179 054 026 O70 144 162 124 115 114 36 099) 42. 
138 169 —009 —134 004 148 —088—008 038 —109 —074 —074 —048 —076 79 113) 43. 
086 150 —032 102 048 223 —068 —002 —048 —042 —023 —087 —003 —034 23 O17) 44. 
j 318 378 180 096 304 259 060 054 052 398 414 174 263 147 54 139) 45. 
118 086 154 034 156 007 378 258 233 119 102 200 262 240 14 120] 46. 
064 096 169 063 063 093 122 O78 220 089 130 142 259 106 40 425] 47. : 
260 186 356 121 334 128 181 O84 282 473 357 299 372 379 28 089] 48. 
147 225 245 154 299 ° 105 093 117-013 300 273 124 135 178 7 —002| 49. 
305 032 188 117 106-103 240 182 008 166 069 163 122 238 18 —220 | 50. 
031 132 216 084 122 073 109 093 122 297 212 170 209 192 PO 007 «(018 118| 51. 
—016 122 226 049 086 021 060-007 310 088 162 169 399 129 —002 —220 118 52. 
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TABLE 3 
ROTATED Factor LoApDINGs 
A B Cc D E F G H I K L M N 3) P Q h? 
Vv N Vz GR LE IF J CR Oo PF AX Or E CF (r) (r) (r) 
2. -15 -17 -14 -02 -04 —.08 .08 —.07 -35 
5. —.08 .02 .26 .24 .06 .09 —.06 .08 —.08 .1§ —.12 6.10 —.08 —.07 =. 28 
re -04 .08 —.03 26 288 —.03 -17 .07 -09 .47 .46 —.04 13 —.13 .68 
9. —.or .16 —.or -24 -OF —.03 -00 06 -40 .18 -O7 -02 -00 +03 10 03 «30 
10. —.08 .06 —.06 —.02 —.06 .08 .o8 -03 .36 —.02 —.08 -00 .08 —.14 .22 
12. —.02 .16 34 —-.06 .17 .03 13 —.05 -38 .34 «.17 —.02 .00 —.1§ —.02 —.02 
13. -06 —.02 —.07 +22 +23 —.03 -O7 -O§ —.14 -03 —.16 —.06 -49 
17. -07 .00 —.10 .32 .43 —.09 =. 05 II 10 16 .47 
18. —.06 .09 .00 —.04 42 —.. 01 -15 24 -09 —.16 “10 —.39 .43 
19. -1§ .04 .02 .03 .09 —.06 -26 .07 —.14 .12 —.08 —.08 —.or -0§ —.33 
20. -19 +07 -08 +02 -18 20 14 -36 .04 —.02 —.17 -34 
21. —.12 .03 .04 .19 .07 —.04 —.04 .09 .23 12 .53 +03 .02 —.12 —~.06 .44 
22. 27 to 03 -06 .03 -00 +17 -07 —.04 -45 -I0 .03 —.13 —.18 .42 
23. —.04 .33 —.08 .36 .02 .o8 .14 .02 —.02 —.10 —.14 .06 .25 
24. —.04 .19 .10 .02 .26 —.05 14 —.09 .20 .12 .04 .26 
25. Ir -16 -13 12 —.07 10) 14 -14 —.04 -44 -02 -07 .07 —.02 -41 
26. —.08 —.02 —.06 —.04 «6.28 32 16 -Ir —.06 -34 -23 —.16 —.20 —.05 -49 
28. -18 .03 -05 -12 -16 .20 6.33 .13 —.10 —.02 +32 290 —.35 -07 -O1 
20. -16 «16 o8 —.07 -16 —.04 18 .02 .14 —.03 .06 .00 —.03 —.12 .45 
30. -08 .05 -23 -25 —.02 Io —.04 .08 .09 —.09 -00 13 —-24 +13 -06 —.08 12 .28 
3I. -28 —.02 —.0§ .04 .46 .08 —.or —.05 -02 -50 
32. —.07 24 —.06 —.02 .07 .090 —.12 —.26 —.03 .38 
33. +04 18 .06 .14 .03 .0§ .0§ —.07 .02 .47 .19 —.02 —.c8 14 —.02 —.13 .38 
34. -08 —.08 22 .02 —.08 —.09 .07 .04 —.10 .39 .35 —.02 —.I5 
35.  .8§ —.83 18 —.07 —.03 12 —.07 —.02 .44 
36. .04 .20 1.00 .12 —.02 —.03 .14 .03 .02 —.03 .90 
37- -73 .0€0 .04 —.03 —.03 —.08 :05 —.02 —.05 .08 —.09 .80 
30. Ir +43 «. 3 «6.16.21 —.06 —.02 -15 -16 .00 —.01 —.04 :07 
40. 14 .14 .28 .46 —.05 O04 —.02 --.07 .03 «32 04 —.0I —.10 .50 
41. -or +03 .18 Ir -06 -It 30 -16 —.02 —.or —.03 
42. .09 .0§ —.07 .02 —.02 .36 .oF 
43. —.16 .08 —.12 —.03 .00 .37 .09 —.18 .04 13 —.07 —.10 .27 .09 .02 .35 
44- —.09 .02 —.02 —.08 —.04 .40 -09 «6.16 —.06 .of —.06 .08 8.23 —.06 .02 -37 
45- —.08 .23 +27 -28 10 —.02 —.or -07 -16 —.o1 -18 .07 -45 
47- —.04 —.04 .03 .04 .09 .12 —.04 .06 —.06 .02 —.20 —.03 —.12 .30 
48. .18 23-19 —.04 «= «47 —. 08 18 .04 —.03 .48 
40. 22 —.03 —.07 .32 14 .30 .09 —.07 —.06 .06 .21 —.08 —.09 —.03 —.02 .06 .33 
50. —-04 .44 +16 .0€0 —.18 .23 .20 —.10 —.09 .32 —.10 —.08 .02 .03 .18 —.18 .56 
51 22 .o8 —.15§ .18 .10 —.02 —.15 .28 —.10 .05 .09 .08 .03 .05 —.22 03 —.04 .30 
$2. -08 .66 —.10 .09 .17 .06 —.0r —.03 +24 +04 .00 —.01r —.04 .08 .§7 


test of symbolic thinking, along with the 
Line Drawing test (8). Both tests had 
their highest loading on originality, al- 
though in the case of the Line Drawing 
test, which had a very low communality, 
the loading is only near-significant (.26). 
Apparently the invention of symbols in- 
volves originality as much as those tests 
that call for clever, remote, and uncom- 
mon responses, 


An interesting paradox may be noted 
in that the Line Drawing and the Symbol 
Production tests were scored on the basis 
of commonness (in the former test) and 
intelligibility (in the latter) of the sym- 
bols. Seemingly, a similar psychological 
process is demonstrated in one type of 
test resulting in uncommon responses, 
and in other tests resulting in responses 
that are common and readily understood. 
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Perhaps the conception of the factor of 
originality needs to be broadened to in- 
clude symbolizing activities. The produc- 
tion of symbols of the kinds involved in 
tests 18 and 1g may be a sufficiently 
uncommon act in itself to require origi- 
nality, so that the scoring can stress com- 
mon responses and yet measure original- 
ity. In any event, Symbol Production 
emerges in this study as a very promising 
test of originality. 


Factor J. Perceptual Foresight (PF) 
g Competitive Planning .40 
34 Planning a Circuit 89 (85 AX) 
12 Route Planning (.34 Vz, AX) 
10 Symbol Grouping 36 
35 Code Analysis -33 (.35 GR) 
17 Match Problems If 32 (.43 AX) 


This factor seems to involve the ability 
to explore visually possible courses of 
action in order to select the most effective 
ones for getting solutions to detailed 
visual problems. Competitive Planning, 
Route Planning, and Symbol Grouping 
were selected or designed as measures of 
foresight. Each of these tests presents a 
task in which, ostensibly, several alterna- 
tive courses may be pursued to the goal. 
Only one of the courses, however, is con- 
sidered to fulfill the criterion given. In 
each case the examinee must look well 
beyond the step at which he is working in 
order to select the proper course of ac- 
tion. The steps or moves that are most 
obvious or seem to provide immediate 
gains are not always the best in terms 
of future gains or in leading to a correct 
solution. 


Factor K. Adaptive Flexibility (AX) 
46 Match Problems 57 
33 Planning Air Maneuvers .47 
17 Match Problems JT -43 (.92 PF) 
34 Planning a Circuit -35 (.39 PF) 
12 Route Planning (.34 Vz, PF) 
50 Mechanical Principles _.32 (.44 Vz) 


This has been identified as adaptive 


flexibility, principally on the basis of 
Match Problems, which served as a refer- 
ence test for this factor. The presence of 
three Air Force planning tests on this 
factor might suggest that this is the Air 
Force planning factor. It is much easier” 
to rationalize these tests as flexibility 
tests, however, than to rationalize Match 
Problems as a planning test. This sug- 
gests that the Air Force planning factor 
was indeed adaptive flexibility. Another 
possibility is that the two factors have 
telescoped into one, but we prefer the 
interpretation stated first. 

Adaptive flexibility was defined in the 
creativity study (13) as the ability to 
change set to meet new requirements 
imposed by changing problems. In an 
unpublished study in collaboration with 
the Institute of Personality Assessment 
and Research at Berkeley there was evi- 
dence of a further generality of adaptive 
flexibility in the significant loadings of 
two tests that required perceptual analy- 
sis and perceptual reorganization prior 
to solution. In that study it was also 
indicated that problem solving requiring 
insights may in general depend on the 
ability to shift set. 

In all the tests in the above list some 
type of visual restructuring in order to 
solve a problem seems to be a common 
feature. Not only is the examinee re- 
quired to do visual restructuring but he 
must also reorganize perceptually the 
same material in different ways to meet 
different requirements. In the Match 
Problems test, some of the problems are 
impossible of solution unless the ex- 
aminee breaks a self-imposed instruc- 
tional set. In Planning Air Maneuvers, 
frequently the same or similar letter 
configurations are used but the differ- 
ences in the start and end positions 
among these configurations require the 
examinee to find new patterns. 
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Factor L. Ordering (Or) 


21 Picture Arrangement 53 
22 Sentence Order 45 
25 Word Matrices 44 


20 Temporal Ordering 36 
31 Seeing Deficiencies 36 (.46 J) 
1 Matrix Order 33 
28 Procedure Applications .32 
41 Inference Test 30 (.32 V, .30 LE) 


This is the hypothesized ordering fac- 
tor. Tests 21, 22, 25, 20, and 1 all re- 
quire some indication of the correct ar- 
rangement of objects or events. The three 
tests constructed or adapted to measure 
the temporal aspect of ordering, Picture 
Arrangement, Sentence Order, and Tem- 
poral Ordering, all had significant load- 
ings. Of the three tests that called for 
the hierarchical arrangement of subject 
matter, Outlining I and II and Word 
Matrices, only Word Matrices was 
weighted significantly. 

In the two leading tests, Picture Ar- 
rangement and Sentence Order, a tem- 
poral arrangement of events is required 
in order to make a sensible story. In 
Picture Arrangement, the events are pre- 
sented pictorially; in Sentence Order, 
the events are in language form. These 
two tests were involved in Adkins’ rea- 
soning study (1). In order to explain 
their significant loadings on a verbal fac- 
tor in that study, it was suggested that 
verbal comprehension would be more 
suitably named verbal relations where 
“perception and manipulation of verbal 
relations” is emphasized rather than just 
the knowledge of word meanings. The 
present analysis, with the increased num- 
ber of ordering tests, has effected a sepa- 
ration between the verbal-comprehen- 
sion factor, defined by vocabulary tests, 
and ordering, defined by tasks in which 
manipulation of relations is required. 

Consideration of the tests listed above 
leads to the interpretation of ordering 
as the ability to arrange objects or events, 
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or to define an arrangement of objects 
or events, in a sequence that is mean- 
ingful either in terms of time, hierarchi- 
cal, or causal relationship. 


Factor M. Elaboration (E) 


7 Effects 47 (46 CF) 
14 Planning Elaboration 44 (.30 J) 
13 Planning Skills II .40 (.31 CF) 
15 Figure Production 37 (.30 O) 
26 Unusual Methods 4 (.32 O) 


This is the elaboration factor hypothe- 
sized to be involved in tasks requiring 
the specification of details that would 
contribute to the development of an 
idea or to the variations of an idea. 

In both Planning Elaboration and 
Planning Skills II, the verbal specifica- 
tion of details was called for in the de- 
velopment of plans or activities. The sig- 
nificant loading of Figure Production 
on this factor indicates that elaboration 
seems to be common to both figural and 
verbal tasks. In the Figure Production 
test, the examinee is required to produce 
a meaningful figure out of an ambiguous 
marking. The figures produced were 
scored for the degree of elaborateness; 
that is, the more detail the examinee 
puts in, the higher his score. 

The significant loadings of the Effects 
test on this factor, a test in which the 
examinee lists four future effects of a 
given present trend, and of the Unusual 
Methods test, which calls for different 
ways of handling given problems, indi- 
cate that idea development can be carried 
on in terms of variations of an idea that 
show its range or scope. 


Factor N. Conceptual Foresight (CF) 
4 Pertinent Questions 58 
7 Effects 
16 Alternate Methods 


11 Contingencies 39 (43 J) 
8 Consequences 
(Remoteness) 30 (.34 O) 


The tests with significant loadings on 


ia 
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this factor seem to require the examinee 
to envision a problem situation in such 
ways that needs or consequences are 
anticipated. In tests 4 and 11 the ex- 
aminee must be aware of or anticipate 
the needs in a situation. In tests 7 and 8 
he must see the more remote conse- 
quences connected with hypothetical 
happenings. 

The leading test on factor N, Pertinent 
Questions, was constructed as an orienta- 
tion task in which the examinee must, 
by asking many different questions, show 
an awareness of the variables present in 
the situation. This was a decision-making 
task in which very little information was 
provided. The task proved to be more 
than an awareness process; the examinee 
apparently conceptualized the variable in 
terms of the needs he foresaw in the 
situation. 

The Effects test was devised as a test 
of extrapolation, and, along with the 
Consequences test, was scored for remote- 
ness. These two tests require the ex- 
aminee to think of various features of 
the situation in terms of which the effects 
or consequences could be indicated. ‘This 
is a task involving foresight. 

Factors O, P, and Q were not defined 
by any loadings greater than .go0, and 
were considered residual factors. 


VI. Discussion 


The formulation of the hypotheses of 
the abilities necessary to planning per- 
formances was made in terms of the 
classes of planning activities and in terms 
of the qualitative aspects of planning. It 
was felt that planning could not be ade- 
quately investigated as a unitary psycho- 
logical function but must be broadly con- 
ceived as a term covering a number of 
classes of activities and entailing a num- 
ber of separate abilities. It was hypothe- 
sized that as planning proceeds through 


certain stages or when different aspects 
of planning are emphasized, different 
abilities are called upon, either consecu- 
tively or in combination. 


A. RELATION OF OBTAINED FACTORS TO 
INITIAL HYPOTHESES 


The hypothesis of orientation as an 
activity necessary not only to planning 
performance but to problem-solving in 
general did not correspond to any one of 
the previously found factors or to any 
new ability. Orientation remains only a 
descriptive term for those activities that 
involve the seeing of an order or trend 
in a mass of information and that show 
an awareness of the pertinent variables 
operating in a situation. Tests of the 
two subhypotheses of orientation are 
loaded variously on factors of ordering, 
originality, eduction of conceptual rela- 
tions, and conceptual foresight. 

The futuristic character of planning 
performance was hypothesized to call 
upon a predictive ability. Tests of predic- 
tion were constructed in terms of two 
variations of prediction: extrapolation 
and foresight. For each of these varia- 
tions, both perceptual and conceptual 
tests were included in the analysis. A 
separation occurred in terms of this dis- 
tinction rather than between extrapola- 
tion and foresight. Competitive Plan- 
ning, Symbol Grouping, and Route 
Planning, designed as perceptual tests 
involving foresight, emerged on per- 
ceptual foresight. 

The finding of a perceptual-foresight 
factor, which is defined in part by maze 
tests, is in accordance with C. D. Porteus’ 
belief that the maze tests he used were 
tasks involving foresight (12). Porteus 
implied that there is more than one level 
of foresight or planning, i.e., that there 
is a low-level kind, as typified by the 
ability called upon in solving the mazes, 
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and a more abstract or high-level kind, 
as called for in creative work, and that 
it is necessary to be capable of the lower- 
level foresight before one can exhibit 
the higher-level type. The results of this 
study would seem to confirm the ex- 
istence of more than one kind of fore- 
sight. Perceptual foresight and concep- 
tual foresight, however, were found as 
distinct factors, indicating that one type 
of foresight is not necessarily basic to the 
other. 

The three conceptual tests constructed 
to measure the predictive ability helped 
to define the conceptual-foresight factor. 
Foresight in this study has been viewed 
as an awareness of possible future events 
that have a relation to a present situation. 
This definition of foresight appears to 
involve the flexibility of ideas, as indi- 
cated by the fact that the examinee has 
to conceptualize different needs that 
might be connected with the given situ- 
ation in order to answer the problem 
in the foresight tests. 

The results do not seem to support 
the subhypothesis of extrapolation as 
one type of task involving a predictive 
ability. The Series test, intended as a 
perceptual task requiring extrapolation, 
was not significantly weighted on any of 
the factors in this analysis. As indicated 
in the interpretation of the conceptual- 
foresight factor, the Effects test and Con- 
sequences could be characterized more 
as tests of foresight than of extrapolation 
because of the additional information re- 
quired in order to give suitable responses. 

Due to the multiple-response character- 
istic of most of the conceptual-foresight 
tests, it is not clear to what extent re- 
sponse fluency enters into the factor. It 
remains for future study to clarify this 
aspect of the factor. 

The identification of a factor of elab- 


oration was based upon the significant 
loadings of the tests connected with the 
subhypothesis of specification. It is an 
interesting finding that elaboration is 
not confined to verbal tests, as is indi- 
cated by the presence of a figural test on 
this factor. Apparently the ability to 
elaborate in a figural task is related to 
the ability to specify details in a verbal 
task. Had there been more alternate 
tests requiring perceptual elaboration, 
however, a factor for such tests might 
have been split off from one for con- 
ceptual elaboration. 

The significant loading of the Effects 
test on this factor indicated that the 
factor should be described not only in 
terms of specifying details in order to 
develop an idea but also in terms of the 
production of variations of an idea which 
serve to show the scope of the idea. 

Some comment should be made about 
the subhypothesis of symbolization. The 
elaboration factor apparently does not 
encompass the symbolic representation of 
ideas; neither does the symbolization sub- 
hypothesis define a separate symboliza- 
tion factor. The expected high correla- 
tion between the two tests, Symbol Pro- 
duction and Line Drawing, did not ma- 
terialize. The variances of these two tests 
were accounted for principally by the 
originality factor. 

The confirmation of an ordering fac- 
tor is indicated by the significant load- 
ings for tests that contained items re- 
quiring the ordering of bits of informa- 
tion in temporal, causal, and logical 
sequences. 

The tests designed to measure think- 
ing ingenuity in relation to planning 
tasks did not serve to determine a fac- 
tor of ingenuity. Much of the variance 
on these tests, particularly that of Un- 
usual Methods, was accounted for by the 
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originality factor. There was a question 
whether originality, defined in terms of 
unconventionality of thinking in the 
creativity study, would be important in 
planning tasks where new procedures, 
or applications of old procedures to new 
situations, must be devised. Admittedly 
the hypothesis of ingenuity was not satis- 
factorily tested in some respects. There 
were too few tests representing this hy- 
pothesis, and it might have been better 
if the experiential features in these tests 
had been reduced. 

Two of the three tests designed to 
measure the evaluation aspects of plan- 
ning emerged on the judgment factor. 


B. RELATION OF RESULTS TO FACTORS IN 
Previous STUDIES 


Twenty tests were included in the 
planning battery for fourteen reference 
factors. Only three of the reference fac- 
tors, associational fluency, symbol ma- 
nipulation, and integration III, did not 
emerge in some form in the analysis. 
Tests of the planning factor found in 
the AAF analyses, and of adaptive flexi- 
bility, isolated in the creativity study, 
emerged together on a single factor 
(adaptive flexibility). This result helps 
to clear up to some degree the question 
regarding the essential ability repre- 
sented by the AAF planning factor. It 
also broadens the conception of adap- 
tive flexibility. 

The results of this study were also 
informative with regard to the natures 
and scopes of certain other reference fac- 
tors. Judgment was found to be weighted 
in various completion-type tests as well as 
tests with a multiple-choice format. The 
factor of ideational fluency seems to ap- 
ply better to tasks in which responses 
are produced under fairly unrestricted 
conditions and involve responses that 


arise from the more superficial aspects 
of situations. 

A new test, Symbol Production, seems 
to provide a good measure of the origi- 
nality factor, previously isolated in con- 
nection with the creative-thinking bat- 
tery, and indicates a possible symbolic 
aspect of originality. 

Elaboration, ordering, perceptual fore- 
sight, and conceptual foresight are re- 
garded as new factors, but we should con- 
sider their possible relations to previously 
known factors. The verification of these 
factors, of course, will depend on future 
research. Elaboration may represent a 
new and more significant dimension of 
ideational fluency. The factor interpreted 
as perceptual foresight might alterna- 
tively be identified as the integration III 
factor found in the AAF analyses, since 
several of the tests that helped to define 
the latter factor were loaded significantly 
on factor J of this study. However, the 
leading test on integration III, Plan- 
ning Air Maneuvers, was included in this 
study but did not have a significant load- 
ing on factor J. In view of this and of 
the fact that three tests designed to meas- 
ure foresight had significant loadings on 
factor J, it was felt that this factor would 
be better characterized as perceptual fore- 
sight. 

Conceptual foresight bears some resem- 
blance to the spontaneous-flexibility fac- 
tor identified in the creative-abilities 
study, but the idea flexibility involved in 
conceptual foresight is more purposive, 
or goal-directed. 


C. RELATION OF RESULTS TO THE 
GENERAL PLANNING TEST 


The Lorge test of Planning Skills (test 
32) was included in the analysis as a 
general planning task. Given a detailed 
account of a practical problem situation 
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the examinee had to define and structure 
the problem for himself, find the critical 
areas, perhaps evaluate the alternate 
methods he thought of, and write down 
all the steps necessary for carrying out 
the procedures adopted. Because of the 
complexity of the task a number of abili- 
ties were considered to be involved. 

The Planning Skills test was treated 
as any other variable in the analysis and 
no special attention was given it in the 
rotations of the axes. Results from the 
preliminary testing with two problems 
(only one problem situation was used in 
the final battery) had yielded an alter- 
nate-forms estimate of about .4o. If this is 
a reasonable estimate, practically all of 
the true variance was accounted for in 
terms of common factors by this study. 
Since the factor loadings are all very low, 
however, our conclusions as to factor 
content are merely suggestive; certainly 
not decisive. 

As expected, the common-factor vari- 
ance represents a number of factors. The 
leading factors and the proportions of 
common-factor variance that each ac- 
counts for are as follows (these propor- 
tions add up to 1.00): 


Factor M. Elaboration 22 
Factor I. Originality 15 
Factor G. Judgment 14 
Factor F. Ideational Fluency .12 
Factor P. (Residual) 18 

(Others) -19 


The Planning Skills test called for con- 
siderable detail in the examinee’s report 
of a plan, hence the elaboration variance. 
Responses were scored to some extent for 
quality, therefore the production of 
clever and uncommon ideas would be 
likely to receive weight and thus intro- 
duce variance in originality. The contri- 
bution of the factor judgment can be ra- 
tionalized by supposing that the exami- 
nee had to select critically the significant 
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features of the problem, to evaluate the 
hypotheses and ideas that came up, and 
to maintain continual check on the in- 
ternal consistency of his plan. The weight 
in ideational fluency probably reflects the 
fact that quantity of ideas was weighted 
in the scoring. As indicated before, these 
small contributions may not be very ac- 
curately estimated. It should also be re- 
membered that the Lorge test represents 
only one kind of planning. 


VII. SUMMARY 


The purpose of this study was to iso- 
late and define the abilities involved in 
planning. A battery of 52 tests covering 
six hypotheses and a number of reference 
factors was administered to 364 USAF air- 
crew trainees. Included in the battery was 
one test that represented a general plan- 
ning activity. The scores were intercor- 
related and 17 factors were extracted. 
Orthogonal rotations resulted in 14 
identifiable factors, ten of which had 
been found previously: verbal compre- 
hension, numerical facility, visualization, 
general reasoning, logical evaluation, 
ideational fluency, eduction of concep- 
tual relations, judgment, originality, and 
adaptive flexibility. Four new factors 
were defined as ordering, elaboration, 
perceptual foresight, and conceptual fore- 
sight. The greatest proportions of the 
common-factor variance of the general 
planning test were contributed by the 
factors of elaboration, originality, judg- 
ment, and ideational fluency. 

It may be said that this investigation 
supports the hypothesis that in planning 
we should find a large number of primary 
abilities involved. Of the factors found in 
this investigation, four are new and seem 
to be unique to planning tests. They are 
the abilities of ordering, elaboration, per- 


ceptual foresight, and conceptual fore- 
sight. 
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